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1. The Government invests in a range of large 
‘scientific facilities to support and develop the nation’s 
science base. Since 2000 the Department of Trade 

and Industry's (the Department) Office of Science 

and Innovation has established new arrangements 

to co-ordinate planning for large facilities. The main 
components are a road map describing large facilities 
which UK scientists are likely to need in the next 10 to 
15 years, and a central fund (the Large Facilities Capital 
Fund) of around £100 million per year to support 
investments in such facilities by Research Councils. The 
Research Councils are the Arts and Humanities Research 
Council (AHRC), Biotechnology and Biological Sciences 
Research Council (BBSRC), Council for the Central 
Laboratory of the Research Councils (CCLRC), Economic 
and Social Research Council (ESRC), Engineering and 
Physical Sciences Research Council (EPSRC), Medical 
Research Council (MRC), Natural Environment Research 
Council (NERC) and Particle Physics and Astronomy 
Research Council (PPARC). 
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2 The Department has earmarked £830 million 
from the central fund to 15 prioritised projects as set 
out in Figure 1 on page 6. Once prioritised, these 
projects progress by presenting science and business 
cases to secure project approval. To date 10 projects 
have received Departmental approval. Part 1 of this 
report evaluates the strength of current processes for the 
identification, appraisal and prioritisation of potential 
investments in large scientific facilities. Part 2 evaluates 
performance in delivering those facilities prioritised for 
support. Ten projects examined in detail in this report are 
outlined in Figure 2 on pages 8 and 9. In total they have 
a capital budget of £1 billion. The study methods are 
summarised at Appendix 1. 


3 __ Ultimately, the value for money to be obtained 
from these facilities will depend on the scientific 
discoveries they help make and the effective exploitation 
of that science. Our study concludes that, though such 
outcomes will always remain uncertain, the current 
arrangements should deliver a significant contribution to 
the development of the nation’s scientific infrastructure. 
The arrangements include the development of a common 
plan known as the road map. This is co-ordinated by 
Research Councils UK — a partnership of the research 
councils — which addresses priorities which cut across 
scientific disciplines. The road map has allowed scientific 
priorities to be considered in a more systematic way 
across disciplines. Working within HM Treasury’s Green 
Book guidance on project appraisal and evaluation, ' 
special attention now needs to be paid to strengthening 
the information available to support choices between large 
facility projects. 


4 __ The projects we examined had generally been 
established in accordance with good practice principles 
and standards as set out in methodology advocated by 

the Office of Government Commerce for managing 
projects, called PRINCE2.? More consistent application of 
that Office’s Gateway reviews to the key stages of every 
project, would ensure all teams benefit from wider sources 
of advice on areas such as project management, project 
costing and funding and procurement options. 


5 It is still early to assess fully the portfolio of projects 
against delivery to time and budget. To date, performance 
against the approved capital budgets suggests some good 
budget management, for example on the James Cook 
research ship, but also projects where forecast capital 
costs already exceed budget even though still at an 

early stage. More significantly, some project teams have 
significantly underestimated the likely running costs of 
facilities once they are delivered. In addition, more work 
is needed by Research Councils to examine the potential 
impact of these facilities on the future demand for research 
funding, their capacity to support any new demand, or 
the effect of doing so on other areas of activity. Full use of 
these facilities will depend on research ideas competing 
successfully for research funding, through peer review, 
against other calls on limited Research Council budgets. 
As the new facilities come on stream, the Research 
Councils will need to monitor the impact on the demand 
for research funding and ensure lessons are learned for the 
appraisal of similar facilities in the future. 


SUMMARY 


6 Our detailed conclusions are as follows: 


Current arrangements identify potential projects over 
a sensible planning horizon, allow prioritisation 
across the science base, and are delivering a 
significant programme of new or replacement large 
scientific facilities. The road map approach was 
pioneered by the United Kingdom and has been 
widely commended and adopted by other countries. 


m Prioritisation and assessment strongly reflect the 


primary policy objective of advancing scientific 
knowledge, but economic potential and possible 
exploitation by industry are less fully analysed. 


| Current arrangements concentrate attention on 


availability of capital funding to build facilities but 
there are significant weaknesses in assessing their 
ongoing costs, and the impact (normally in future 
spending review periods) of meeting those costs on 
the balance of Research Council funded activities. 


m Atprioritisation estimates of costs and assessments 


of benefits are preliminary, yet priorities are not 
reviewed if costs or benefits are significantly revised 
as business cases are prepared. Opportunities to 
maximise the overall science benefits of the portfolio 
of projects may thus be missed. 


m Procurement strategies have been adapted to 


the particular circumstances of each project. 
Future projects can benefit from better sharing of 
experience and lessons learned. 


m= More attention needs to be given to specifying from 


the start how the success of individual projects will 
be assessed and measured, drawing on examples 
from some current and existing projects. This should 
enable a fuller assessment of value for money to 

be made once facilities are operational and inform 
appraisal of future potential projects. 
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Recommendations 


1 The Office of Science and Innovation, Research 
Councils UK and individual Research Councils should 
strengthen project appraisal by: 


m ensuring the production of more consistent estimates 
of costs and assessments of benefits at the initial 
point of prioritisation; and 


m ensuring that if a project’s expected costs or benefits 
at business case are significantly different from those 
initially anticipated, its priority is reconsidered at the 
next available opportunity. 


2 The Office of Science and Innovation, Research 
Councils UK and individual Research Councils should 
give greater attention to the future financial sustainability 
of projects. Project proposals should be based on realistic 
estimates of their ongoing costs, the sources of funding 
available to cover those costs and any implications for 
other activities funded by Research Councils. 


3 Research Councils UK should ensure that the road 
map differentiates projects where there is a choice of 
location from those where no such choice is practically 
available. Research Councils UK and the Office of Science 
and Innovation should provide Research Councils with 
guidance to aid preparation of comparisons of different 
locations where a choice is available. 


4 _ To improve the transparency of investment decisions, 
and provide a better opportunity for scrutiny or challenge 
by scientific and industrial stakeholders, Research 
Councils UK should publish the outcomes of and rationale 
behind the prioritisation of proposals as part of the 

large facilities road map. The rationale should include 
commentary on the implications for the overall research 
programme of supporting the construction and operation 
of prioritised projects. 


5 The Office of Science and Innovation, Research 
Councils UK and individual Research Councils should: 


m ~~ ensure an evaluation of the nature and scale of 
the economic impacts derived from building and 
operating large scientific facilities, once they have 
been brought into service, is undertaken; and 


m _ provide guidance to project teams on assessing and 
presenting anticipated economic impacts of large 
facility proposals. 


EXECUTIVE SUMMARY 


6 The Science and Technology Facilities Council, 
which will be established in April 2007, should: 


m= =promote awareness of knowledge and lessons 
from planning, procurement and operation of large 
scientific facilities by bringing project teams or 
members together to share experiences and training; 


mw develop and promote a consistent means of applying 
the science performance management framework 
developed by the Office of Science and Innovation in 
2005 to large facilities planning and operation; and 


mw use its own skills base and partnerships with 
external providers to improve other Research 
Councils’ access to high-grade project management 
skills for large projects. 


7 The Department of Trade and Industry should work 
with Research Councils to ensure the Government-wide 
Gateway review process is applied to large facility projects 
consistently and with a level of independence appropriate 
to their assessment of risk. 


8 Large facility project teams should build on 
procurement lessons from previous projects to secure 
improvements in value for money. Across the portfolio of 
projects there is scope: 


m to undertake a deeper analysis of risks so that 
project teams can separate those which should be 
transferred to a contractor and those which should 
be retained; 


m to make greater use of incentives to encourage 
the timely delivery of key components or 
project milestones; 


m to extend the use of pain/gain share conditions 
in contracts, thereby increasing the incentives for 
contractors to contain costs; 


m to improve the packaging of work by considering 
the separation of those elements where there is 
a limited pool of potential suppliers from less 
demanding elements; and 


m= for more active promotion of the work on offer 
to potential suppliers who might otherwise be 
deterred from bidding by the scientific nature of 
the overall project. 
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Ea | The National Audit Office looked at ten projects in detail’ 
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Diamond Synchrotron 
(Diamond Light Source Ltd) 


Diamond is a 24 cell, 

3 giga electron volt, 3rd 
generation synchrotron light 
source producing intense 
x-rays and shorter wavelength emissions for research in the 
biological, physical, environmental and engineering sciences. 
The synchrotron is being built by, and will be operated by, a joint 
venture company Diamond Light Source Ltd, partly owned by the 
Council for the Central Laboratory of the Research Councils and 
partly by the Wellcome Trust. 


Location: Harwell Science and Innovation Campus, Oxfordshire. 


Budget and Funding: £383.2 million for Phases | and Il, with 
£308.6 million from the Large Facilities Capital Fund. 


Delivery: Phase |, including the first seven beamlines, is due to 
begin operations in January 2007 and Phase Il, including the next 
15 beamlines, is due to be‘completed in 2011. 


Royal Research Ship James 
Cook (Natural Environment 
Research Council) 


The RRS James Cook is a 
replacement for the RRS 
Charles Darwin and is 
sponsored by the Natural Environment Research Council. Its users 
will be marine scientists based, for example, at UK universities 
and the Research Council’s National Oceanographic Centre 

in Southampton. 


Location: Worldwide but mainly Atlantic waters — built in Poland 
and Norway. 


Budget and Funding: £40 million, of which £25 million will come 
from the Large Facilities Capital Fund. 


Delivery: The ship was delivered to the National Oceanographic 
Centre in August 2006. 


ISIS Neutron Source, Second 
Target Station (Council for the 
Central Laboratory of the 
Research Councils) 


The ISIS Neutron and Muon 
Scattering Facility is the 
most powerful neutron producer of its kind in the world. The first 
phase of the project involves supplementing the existing facilities 
with a second target station and the installation of a first suite of 
instruments. It will enable the ISIS science programme to attract 
new users from the key research areas of soft matter, advanced 
materials and bio-science. 


Source: National Audit Office 


NOTES 


1 Project summaries set out the position as at autumn 2006. More detail on each of the ten pak is oreidets in ‘Apponalca 3. oe 
2 £67 million for the redevelopment of the Institute for Animal Health is being provided by the Ree Ss Environment, er aa Rural Acs. 
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Location: Harwell Science and Innovation Campus, Oxfordshire. 


Budget and Funding: £145.6 million for the first phase, with 
£127.9 million from the Large Facilities Capital Fund. 


Delivery: The experimental programme is set to begin in 
October 2008. 


Energy Recovery Linac 
Prototype (Council for the 
Central Laboratory of the 
Research Councils) 


The Prototype is phase one 
of the 4th Generation Light 
Source (4GLS) project. The project will use free electron lasers and 
synchrotron radiation covering the terahertz to soft X-ray energy 
regimes for ultra fast dynamic studies of matter. The first phase 

has been designed to address some of the principal technical 
challenges that would be faced in a full 4GLS facility. 


Location: Daresbury Science and Innovation Campus, Cheshire. 


Budget and Funding: £21.3 million, with £10.1 million from the 
Large Facilities Capital Fund. 


Delivery: Full operational energy recovery by April 2007. 


Halley VI Antarctic Research 
Station (Natural Environment 
Research Council) 


The project involves the building 
of the Halley VI Antarctic 
research station and the 
removal of the existing station, Halley V. Halley provides a unique 
location for monitoring climate, ozone and space weather and 
forms a key part of the UK’s regional presence. The primary users 
of Halley VI will come from within the British Antarctic Survey, an 
institute of NERC. 


Location: Antarctic Ice Shelf. 


Budget and Funding: £34.7 million for both construction of Halley 
VI (26.2 million) and decommissioning of Halley V (8.5 million). 
The Large Facilities Capital Fund is providing £20 million 

for construction. 


Delivery: Delivery of Halley VI and decommissioning of Halley V 
by end of 2009-10 Antarctic summer. 


High End Computing 
Terascale Resource (HECToR) 
(Engineering and Physical 
Sciences Research Council} 


HECTOR is the next generation 
of high performance computer. 
It is the responsibility of Engineering and Physical Sciences 
Research Council and will succeed the CSAR and HPCx computer 
services. Users will span several fields of science including 
computational chemistry, physics and climate modelling. 


Location: dependent on tenderers’ proposals. 


Budget and Funding: £65 million in total; £52 million from the 
Large Facilities Capital Fund. 


Delivery: Phase | scheduled to start in September 2007. 


Muon lonisation Cooling 
Experiment (MICE) (Particle 
Physics and Astronomy 
Research Council is the lead 
council. The experiment is 
hosted by the Council for 
the Central Laboratory of the 
Research Councils) 


The Muon lonisation Cooling Experiment is a step towards 

the possible creation of a neutrino factory which would aid 

the understanding of the properties of neutrinos — one of the 
fundamental particles which make up the universe. MICE seeks to 
demonstrate that “muon cooling” — making a tightly focused muon 
beam — is possible through a process of ionisation. 


Location: Harwell Science and Innovation Campus, Oxfordshire. 


Budget and Funding: Phase | of MICE will cost £22.7 million. Of 
this, the UK will fund £9.7 million, of which £7.5 million will come 
from the Large Facilities Capital Fund. 


Delivery: Phase | is set to be complete by November 2007. 


Laboratory of Molecular 
Biology (Medical Research 
Council) 


The Laboratory of Molecular 
Biology opened in 1962 
and is acknowledged as 
one of the world’s leading biochemical laboratories with users 
from the fields of immunology, cancer biology and biotechnology. 
The LMB project will provide a new, modern laboratory building 
on the current hospital campus. 


Location: Addenbrooke’s Hospital Site, Cambridge. 


Budget and Funding: £155 million, of which £67 million will 
come from the Large Facilities Capital Fund. 


Delivery: building due to be available May 2011. 


EXECUTIVE SUMMARY 


Institute for Animal Health 
(Biotechnology and Biological 
Sciences Research Council) 


The Institute is responsible 

for research, diagnostics and 
surveillance on epizootic (fast 
spreading) viral diseases of farm animals. The project involves 

building a new laboratory for the Institute’s staff and employees 


of the Virology Department of the Veterinary Laboratories Agency 
(part of the Department for Environment, Food and Rural Affairs). 


Location: Pirbright, Surrey. 


Budget and Funding: Current approved cost is £121 million with 
£31 million from the Large Facilities Capital Fund.* 


Delivery: The main laboratory building is scheduled for delivery in 
December 2009. 


Research Complex (Medical 
Research Council) and 
Essential Infrastructure S5SS255) 
(Council for the Central & = Lely 
Laboratory of the Research Sra rama 


Councils) 


lee 


The project will provide a research laboratory, hostel 
accommodation and other infrastructure to enable scientists to 
make optimum use of the Diamond Synchrotron, ISIS and other 
facilities at Harwell. 


Location: Harwell Science and Innovation Campus, Oxfordshire. 


Budget and Funding: £33.5 million for the Complex and 
infrastructure, with £32.4 million from the Large Facilities 
Capital Fund. 


Delivery: The main element of the infrastructure programme — a 
new hostel for visiting scientists — was delivered in July 2006. The 
Research Complex is set for completion in June 2009. 
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PART ONE 


1.1. This Part examines the effectiveness of the 
arrangements put in place to choose between potential 
investments in large scientific facilities. It focuses on: 


m responsibilities for investment; 


m processes for prioritising and funding 
potential investments; 


evaluating scientific benefits of proposals; 
evaluating economic benefits of proposals; 
evaluating costs of proposals; 


handling international collaborations; and 


managing the demand for funds. 


Responsibilities for investment in 
large scientific facilities 


1.2 The Office of Science and Innovation, part of the 
Department of Trade and Industry (the Department), is the 
lead body responsible for UK science policy. The Office 
and the eight Research Councils aim to strengthen the 
UK’s science base, and maximise its contribution to UK 
economic development. The Research Councils are the 
main public investors in civil research in the UK. Each 
Research Council is responsible for deciding the research 
priorities within their particular field, in line with overall 
policy set by the Office of Science and Innovation. 
Priorities cutting across scientific disciplines are addressed 
by Research Councils UK (Figure 3). Research Councils 
UK Is a strategic partnership led by an executive group 


made up of the chief executives of each Research Council. 


The group's primary purpose is to define the overall 
strategic framework for research, training and knowledge 
transfer funded by the Research Councils, and to provide 
input into the wider strategy for the whole science budget. 
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Prioritising investments In 
large scientific facilities 


1.3. Increasingly, the facilities used by scientists in many 
disciplines have become larger and more complex. This 
has necessitated levels of capital investment beyond the 
normal means of a single research institution and, in 

the case of the largest facilities, requiring cooperation 
amongst a number of countries. Failure to make such 
investment risks loss of scientific leadership and the 
international standing of UK science. Within the UK, 
planning and providing access to large facilities is the 
responsibility of the Research Councils. 


1.4 Since 2000, the Department has significantly 
increased investment in large scientific facilities. As shown 
in Figure 1, fifteen projects with a total capital budget of 
£2,230 million are currently committed and/or earmarked 
for support of £1,140 million from the Department and 
Research Councils, with a further £70 million coming 
from other UK public bodies. Contributions from 
charitable institutions or other countries make up the bulk 
of the remaining funding. 


1.5. In July 2006 the Department announced that a new 
Science and Technology Facilities Council is to be created 
by a merger of the Council for the Central Laboratory of 
the Research Councils (CCLRC), the Particle Physics and 
Astronomy Research Council (PPARC) and the nuclear 
physics responsibilities of the Engineering and Physical 
Sciences Research Council (EPSRC). This new body, which 
will be established in April 2007, is intended, amongst 
other things, to create a more integrated approach to large 
facilities, including international negotiations for long term 
projects involving several countries.° 


Director General of Science 
and Innovation 


Research Councils UK 


Council for 
the Central 
Laboratory of 
the Research 
Councils 


(CCLRC) 


Biotechnology 
and Biological 
Sciences 
Research 
Council (BBSRC) 


Arts and 
Humanities 
Research 
Council (AHRC) 


Engineering 
and Physical 
Sciences 
Research 


Council (EPSRC) 


Natural 
Environment 
Research 
Council (NERC) 


Medical 
Research 
Council (MRC) 


: Source: Office of Science and Innovation and the National Audit Office 


Processes for prioritising and funding 
large scientific facilities 


1.6 Prior to 2000 proposals for investments in large 
scientific facilities had been prepared and submitted to the 
Department by individual Research Councils as part of the 
spending review process. There was no explicit mechanism 
for deciding priorities between the various bids. Since 2000 
the Office of Science and Innovation and the Research 
Councils have worked together to draw up and prioritise 

a road map of new or replacement facilities which United 
Kingdom scientists will need access to over the next 10 to 
15 years. In addition, the Office of Science and Innovation 
introduced a Large Facilities Capital Fund in 2002-03 to 
help fund projects judged to be of sufficient priority. This 
supplemented funds already allocated to Phase | of the 
Diamond Synchrotron. By 2003-04, the total funds available 
to Diamond Phase | and other large facilities through the 
Fund amounted to around £100 million per annum and 
funding has continued at this level. 


Office of Science and Innovation 


PART ONE 


; and the Office of Science and Innovation are responsible for investments in large scientific facilities 


Minister for Science and Innovation 


Chief Scientific Advisor and 
Head of Office of Science 
and Innovation 


Economic and 
Social Research 


Council (ESRC) 


Particle Physics 
and Astronomy 
Research 
Council (PPARC) 


1.7. The road map is intended to include projects 
fulfilling one or more of the following conditions: over 
£25 million capital cost; representing a high proportion 
of an individual Research Council’s budget; serving 

the research communities of more than one Research 
Council; or having an international dimension. The 
road map was first published in 2001 and was updated 
in 2003 and 2005. The road map is prepared on the 
basis of submissions from individual Research Councils. 
Projects are subsequently prioritised by the research 
directors group of Research Councils UK, which makes 
recommendations via that body’s executive group to the 
Director General of Science and Innovation. The Office 
of Science and Innovation considers the recommended 
priorities and seeks to earmark available resources from 
the Large Facilities Capital Fund to priority projects. 
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PART ONE 


1.8 The road map, and the criteria for its assembly, 
have been successful in capturing proposals from across 
the science base. Proposals from all Research Councils, 
except the Arts and Humanities Research Council, 

are included in the latest road map and serve a wide 
range of scientific disciplines ranging from astronomy 
to oceanography and the social sciences. Of the 62 
senior scientists working in the public or private sectors 
responding to a questionnaire prepared by us (see 
Appendix 1), only four identified projects which they felt 
should have been captured by the road map but which 
had been omitted — with animal testing facilities, central 
open access materials testing facilities, ultra-high field 
Nuclear Magnetic Resonance and space projects being 
mentioned. A further two scientists believed computer 
facilities — which are in the road map — should have 
been considered as proposals customised or dedicated 
to specific user communities, for example to perform 
biological modelling and computation. 


1.9 The road map approach has been commended by 
evaluative reviews by the United States National Science 
Foundation as well as the Australian and Canadian 
governments,* and a road map has been developed by 
the European Strategy Forum on Research Infrastructure 
(ESFRI)° describing the scientific needs across Europe for 
large research infrastructures for the next 10 to 20 years. 
In particular, the studies cited above have commended 
the road map process as a vehicle for decision-making, 
including analysis of scientific opportunities and 
objectives for large facilities. 


1.10 The 2005 road map was published as an 
unprioritised list of proposals, with commentary on the 
criteria which would be applied to prioritise that list. 

The outcomes of prioritisation and earmarking were 

only publicised once the recommendations from those 
processes had secured support from the Department. Our 
interviews with research directors of Research Councils, 
and Office of Science and Innovation, indicated that these 
key players were fully engaged with and aware of the 
process and rationale behind the prioritisation outcomes. 
But the wider scientific and industrial communities did 
not have the opportunity to scrutinise or challenge the 
prioritisation before the earmarking of funds. 
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Evaluating scientific benefits 


1.11 The Office of Science and Innovation expects 
Research Councils putting forward bids for new facilities 
to submit proposals to the executive group of Research 
Councils UK setting out the scientific justification for the 
proposed facility supported by details. of the indicative 
costs. In practice, each Research Council will have carried 
out its own review of bids within its particular field prior 
to submitting its preferred options to the executive group 
of Research Councils UK. Current guidance from Research 
Councils UK is that proposals should show timescale, 

cost estimates and funding profile, and should address the 
following criteria: 


m importance of science knowledge delivered; 


m contribution to international position of UK science 
and science strategy; 


m timeliness — including impact of delay; 
mw breadth of science base that will benefit; 


® opportunity for knowledge and technology transfer 
or wider benefits; 


m scope for partnership with other funders; 


scope for training and production of trained people. 


The role of Research Councils UK is to collate individual 
proposals for large facilities into the road map, and then 
to prioritise them using these criteria. The criteria map 
onto the core Office of Science and Innovation objectives 
of strengthening the science base and maximising its 
contributions to economic development and quality of 
life. The strongest weighting is given to the contribution 

to strengthening of the science base. This emphasis was 
reflected in the perceptions of the scientists responding 

to our questionnaire. Forty five of the 62 senior scientists 
felt that contribution to basic scientific knowledge was the 
most important factor in selecting projects for investments. 
A further 12 felt that contribution to the international 
standing of UK science was most important. 


1.12 Our work suggested that whilst the prioritisation 
process focused on science benefits, Research Councils 
generally needed to do more to meet the requirements 

of HM Treasury’s Green Book and consider the full range 
of potential impacts that a new large facility might have. 
Large facilities with large capacities imply, in some 
instances, that more or larger research teams will have to 
use the facility if its full capacity is to be utilised. Of the 
ten projects earmarked for funding up to 2003, the nature 
of the research communities likely to use the new facilities 
were specified, but the current size of those communities, 
or the prospective increase in demand, on which the scale 
and scope of the proposal was based, were generally not 
set out. Similarly for the replacement or the refurbishment 
of an existing facility, the factors driving the size of the 
replacement, such as the numbers of internal and visiting 
staff it needed to accommodate, were generally not 
specified in any detail. 


1.13 Once a project has been prioritised, the project team 
must prepare a Case setting out the project's scientific 
value. The science case must be endorsed by Research 
Councils UK before the project goes on to prepare a 
business case and seek formal approval. At the end of 
2003, the Office of Science and Innovation enhanced 
procedures and required all new projects prioritised for 
support to arrange an independent review of their science 
case. The criteria to be considered in the science case 

are the same as those for initial proposals. By autumn 
2006, three projects - HECToR, Diamond Phase II and the 
Research Complex — had presented their science cases to 
the Research Councils UK executive group since this new 
requirement was introduced. These cases described in 
greater detail the areas of science which will benefit, and 
statements of the current demand for comparable facilities 
were included for HECToR and Diamond Phase II. But 
forecast levels of demand, and the assumptions behind 
those forecasts, were still not specified. 


Evaluating economic benefits 


1.14 The Office of Science and Innovation and Research 
Councils are increasingly wishing to maximise the 
economic impact of their activities including the use 

that is made of large facilities. In May 2005, the Council 
for the Central Laboratory of the Research Councils’ 
delivery plan® identified active marketing of the research 
capabilities of its major facilities to businesses as a key 
theme. This theme was reinforced in its March 2006 
neutron strategy’ which announced further steps to 
broaden industry access to large facilities. The case for 
establishing the Science and Technology Facilities Council 
is based partly on the argument that a more integrated 
management structure will maximise the economic impact 
of public investment in this area.® 
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1.15 Regional bodies, including the regional development 
agencies, are increasingly advocating or promoting the 
location of new large scientific facilities in their regions as a 
means of assisting regional economic performance. To date, 
most of the projects underway have offered little choice 

in terms of where they are located, for example a number 
have involved extending existing facilities. Where projects 
have offered the possibility of a choice of location, the 
supporting options analysis has either been insufficiently 
independent or its lateness has delayed the decision-making 
process. For the Diamond Synchrotron, for example, 
approved before the introduction of the road map and 
Large Facilities Capital Fund, the assumption that the facility 
would be located at Daresbury meant that analysis of 
options occurred late in the design process and delayed the 
decision to proceed. For new facilities, or even wholesale 
redevelopment of existing facilities, there may be a realistic 
choice of potential locations. To ensure appropriate analysis 
is undertaken at an early stage, the development of the road 
map in the future should identify those facilities where an 
appraisal of the potential location options will be required 
as part of the formal approval of the business plan. 


1.16 The analysis of potential economic benefits supporting 
bids for new facilities is challenging but needs to be further 
developed. The proposals we examined did not identify 

all the main sources of economic benefit although specific 
opportunities were identified. Local impacts, such as the 
effect on the local economy and local businesses during 
the construction phase are reasonably straightforward 

to measure. It is more difficult to assess the longer term 
economic impact of a new facility: the science is by its 
nature uncertain and the economic benefits can be difficult 
to estimate. Internationally, there have been few evaluations 
of the extent to which advances in scientific knowledge 
supported by large facilities in general, or a particular large 
facility, are converted into commercial innovation, or the 
extent to which the benefit of that innovation accrues to 

the country where the facility is located. The Council for 
the Central Laboratory of the Research Councils’ neutron 
strategy, published in 2006, has acknowledged this gap in 
relation to the benefits of hosting international facilities, and 
has proposed a study to begin to examine some of these 
potential impacts. Previous work in this area has considered 
only the relative merit of the UK hosting an international 
facility rather than contributing to the same facility in 
another country. Extending this work to address the nature 
and absolute level of economic impacts of large facility 
investments, once they have been brought into service, 
could provide a better and more consistent framework 

for the presentation and assessment of competing facility 
proposals in the future. 
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1.17 Such analysis of economic impacts as is currently 
undertaken relies heavily on assertions of potential for 
direct industrial use, industrial use through industry- 
university collaborations, or knowledge transfer 
opportunities. These assertions originate from proposers’ 
knowledge of current industrial interest and opportunity 
and, for some of the projects we reviewed, these were 
tested within individual Research Councils by presenting 
the proposal to a panel or committee representing 
industrial interests. Neither the large facilities road map 
as a whole, nor the prioritisation of projects within it, is 
the subject of direct consultation with bodies representing 
industrial interest in Government science policy. 


Evaluating costs 


1.18 To date the road map has included indicative capital 
costs for each project. In some cases, at this early stage, 
estimates can be relatively basic as they are not based 
upon detailed designs of the proposed facilities. The 
estimates are used to “earmark” funds to projects but 
these funds are not committed until a proper business 
case has been approved by the sponsoring Research 
Council, Research Councils UK executive group and 

the Office of Science and Innovation. Almost inevitably, 
some of the initial estimates used to earmark funds to 

the projects prioritised through the 2003 roadmap have 
proved optimistic. For example, between earmarking 

and business case the estimates of the capital costs of 

the Laboratory of Molecular Biology rose by 55 per cent 
to £155 million as more detailed designs of the facilities 
were prepared. The additional cost will be funded by the 
Medical Research Council which will own the facility. 

In a second case — the redevelopment of the Institute for 
Animal Health’s laboratories including incorporation of 
the Virology department of the Veterinary Laboratories 
Agency — cost estimates varied, rising by up to a third, 

as the project was developed. In 2006 it was decided 
that the original funding of £121 million, specified when 
the project was earmarked in 2004, would be used to 
build a smaller facility, which will house 25 per cent less 
staff than intended in 2004 and which will no longer 
include a research hotel for visiting researchers. The main 
laboratory building is now due to be completed by the 
end of 2009, a year later than envisaged when the project 
was prioritised. The project’s principal users — the Institute 
for Animal Health and the Department for Environment, 
Food and Rural Affairs - have confirmed that the rescoped 
facility will meet their needs. 
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1.19 Whilst it is right that the business case and revised 
costs should form the basis for holding project teams to 
account for delivery, there is currently no mechanism for 
reviewing the original prioritisation should the business 
case cost or proposed project scope and benefits differ 
significantly from the original proposal in the road map. 
There are a number of potential consequences should the 
initial estimate of cost, and thus the earmarked allocation, 
prove to be too low: 


The basis for prioritisation and allocations from the 
Large Facilities Capital Fund could be distorted by 
the uncertainties in the cost information available 
at earmarking. 


m Projects could be delayed, and substantial 
amounts of project team and management time 
could be expended, trying to fit projects within the 
original earmarked allocation or trying to obtain 
increased funding. 


m@ If a project experiences cost pressure after 
earmarking but is unable to obtain further funding, 
or de-scope appropriately, there is a risk that the core 
science objectives of a project may not be achieved. 
We are not aware of this risk maturing on any of the 
projects to date. 


Handling international collaborations 


1.20 The projects earmarked for funding prior to 2005 
were primarily national facilities, with the exception of 
the £22.7 million Muon lonisation Cooling Experiment 
(Phase |) which is hosted in the UK but being taken 
forward by an international collaboration of scientists. 
The 2005 road map argues that in many circumstances 
the UK’s interests would be best served by participating 
in a facility overseas, for example through international 
subscription or bilateral arrangements with the host 
country. As the cost of building large facilities increases, 
the Research Councils expect that co-operation at an 
international level will become increasingly necessary 
if projects are to be affordable. Twelve of the 20 new 
projects in the 2005 road map potentially involve 
extensive elements of international collaboration. 


1.21 The evolution of international collaborations is less 
predictable or controllable than projects hosted by the UK 
and primarily serving UK scientists. Indeed the balance 

of costs and benefits of these collaborations for the UK is 
likely to change significantly during their evolution. It will 
therefore be increasingly important to regularly review the 
priorities for international projects rather than setting them 
once and reviewing only on an exception basis as currently. 


1.22 We reviewed the assessment of international options 
in projects supported so far. Providing access for UK 
scientists via other countries’ facilities, or establishing 
research prototypes outside the UK, was not considered 
in some cases because it would relinquish an important 
UK presence (Halley Antarctic research station), involve 
the wholesale emigration of an existing UK research 
institute (Institute for Animal Health, Laboratory of 
Molecular Biology), or lose the advantages of co-location 
with other UK facilities which formed the core rationale 
(Research Complex). In others it was not pursued because 
it would take too long to broker agreement (HECToR), 
lose scientific leadership (Muon lonisation Cooling 
Experiment, ERLP), fail to yield sufficient reliable access 
to meet forecast UK demand (RRS James Cook, Diamond 
Synchrotron), or a combination of these and excessive 
cost (ISIS second target station). 


1.23 We do not challenge the over-riding nature of the 
generally qualitative reasons for rejecting use of other 
countries’ facilities in these cases. Nevertheless as more 
new facilities go beyond the capacity of individual nations, 
the choice between providing a facility in the UK scaled to 
meet demand from UK scientists, and contributing to the 
construction of an international facility in another country 
scaled to meet international demand, are likely to become 


_ Percentage of fund earmarked 
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more complex and finely balanced. In such circumstances 
more detailed analysis of the costs and benefits of these 
alternatives may be required than has been evident — or 
necessary — for the projects approved so far. 


Managing the demand for funds 


1.24 Demand for new funds remains strong despite the 
extent of allocations already made. The number of new 
project proposals in the road map has risen from 14 in 
2001, to 20 in 2005, five of which were prioritised for 
funding. Following prioritisation of the 2005 road map, 
and earmarking of resources based on that prioritisation, 
the Large Facilities Capital Fund was fully allocated for 
four years to 2009-10 (Figure 4). 


1.25 The lack of short to medium-term headroom results 
firstly from the level of demand as the fund has developed, 
and secondly from decisions to earmark 100 per cent of 
funding several years ahead. The continuing gap between 
funding availability and investment aspirations increases 
the importance of requiring consistent project proposals 
which clearly address prioritisation criteria and facilitate 
the difficult but necessary choices between very different 
types of investment. 


ation 0 the 2005 road map, the Large Facilities Capital Fund was fully earmarked to 2009-10 
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| Source: National Audit Office analysis of Research Council UK data 


2011-12 2012-13. 2013-14 2014-15 2015-16 


aa he National Institute of Medical Reach for which the Medical Research Council requested £140 million 
veer) the fice of Science and Innovation decided to await development of the Institute’s business case 
t from the Large Faaiities 3 Fund. This project is therefore not reflected in the above diagram. 


BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES 15 


PART TWO 


2.1 Once projects have received earmarking and their 
science case has been reviewed, the project teams 
established by the host or sponsoring Research Council 
take the proposals forward. This Part examines the 
effectiveness of the arrangements put in place to prepare 
business cases ahead of project approval and subsequently 
manage the delivery of projects. It focuses on: 


mw the application of the Office of Government 
Commerce's Gateway review process to projects of 
this type; 


m _ the acquisition and application of project 
management skills; 


m _ the specification of project outcomes and outputs in 
business cases; 


m __ the delivery of projects against approved budgets 
for capital costs and costs in-use and approved 
completion dates; 


m __ the procurement strategies adopted by teams; and 


the plans made for operating the facilities. 


Using the Gateway review process 


2.2 All earmarked projects are required by the Office 

of Science and Innovation to be reviewed in line with 
Government-wide guidelines for the Gateway review 
process. Figure 5 summarises the process. As at autumn 
2006, six of the projects we visited had proceeded beyond 
Gateway 3 (the investment decision), and thus substantial 
funds had or were being committed. The other five 
projects had at a minimum proceeded beyond Gateway 

1, the point at which business cases are reviewed. 
Summaries of the ten projects are provided at Appendix 3. 
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Delivering projects 


2.3 Projects have generally benefited from Gateway 
reviews with review teams making actionable 
recommendations to assist teams in areas such as 

project management, project costing and funding and 
procurement options. The RRS James Cook project was 
encouraged to adopt an output based specification at 
Gateway 1 — an approach it was already considering — and 
doing so has helped the project avoid cost increases due 
to design flaws which are the shipyard’s responsibility. 

The HECTOR project addressed critical issues regarding 
the phasing of funding and the handling of depreciation 
following recommendations from the Gateway 1 review 
team. The Diamond team produced updated costings for 
the second phase of the project shortly after the Gateway 2 
review recommended greater attention to this phase. 


2.4 Not all projects, however, have been subject to 
appropriate Gateway reviews at relevant stages. The 
Laboratory of Molecular Biology project has not been 
subject to a separate Gateway 1 review. The project team 
after consulting the Office of Science and Innovation 
decided to combine the Gateway 1 and 2 reviews in order 
to expedite project delivery. The Energy Recovery Linac 
Prototype business case has not been subject to Gateway 
review in its own right, even though the Prototype was 
estimated to cost £12.9 million when it was approved. The 
project team in this case viewed the Prototype as a stage 
of the bigger 4th Generation Light Source from which it 
was derived and which was subject to a Gateway 1 review 
in 2002. In general the composition of review teams 

was consistent with Office of Government Commerce 
guidance. However, the Halley VI project has relied on 
internal reviews led by the project board, until Gateway 3, 
which was undertaken by a team external to both the 
board and the project's sponsoring research institute. For 
projects, such as Halley, which are judged by project 
teams and or Research Councils as low risk, Office of 
Government Commerce guidance still recommends that 
review teams should be independent of the project. 


Projects are required ic 0 through the Gateway 
review process 


Stage reached 
for each project 
by autumn 2006 


Gateway 0 - Science Case! 


Considers the justification for the project 
based on the scientific objectives and 
views of the relevant scientific community. 


Gateway 1 — Business Case 


Considers the justification for the project 
based on business needs and an 
assessment of the project's likely costs and 


Research 


potential for success. Complex 


Laboratory 
of 
Molecular 
Biology? 


ERLPS 


Gateway 2 — Procurement Strategy 


Assesses the project's viability, its 
potential for success and whether the 
project is ready to invite proposals or 

tenders from the market. 


Institute 
for Animal 


Health 
Gateway 3 — Investment Decision 


Confirms that the recommended 
investment decision is appropriate before 
the contract is placed with a supplier 
or partner. 


ISIS 2 
Diamond 
MICE 
Halley VI 
HECToR 


Gateway 4 - Readiness for Service 


Focuses on whether the solution is 
robust before delivery and the basis for 
evaluating ongoing performance. 


RRS James 
Cook 


Gateway 5 - Benefits Realisation 


Focuses on ensuring the project delivers 
the benefits and value for money 
identified in the business case. 


Source: National Audit Office drawing on Office of Government 
Commerce and Research Councils UK documents 


NOTES 
1 Gateway 0 - the “science case” — is an Office of Science and 
Innovation requirement (see paragraph 1.13) rather than the Office of 


Government Commerce Gateway 0, which is a review of a programme of 
related projects. 


2 Project has not been through a separate Gateway | - a combined 
Gateway 1 and 2 review is planned. 


3) ERP has not been through the Gateway process in its own right — but 
the project team anticipate outcomes of the prototype feeding into the 
Gateway 2 for the bigger 4th Generation Light Source project. 
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Project management 


2.5 Project teams have been established in accordance 
with good practice principles and standards as set out 

in the PRINCE2 methodology advocated by the Office 

of Government Commerce for managing projects. This 
included governance arrangements, the appointment of a 
senior responsible owner, access to professional advice, 
establishment and maintenance of project documentation, 
and control procedures. Many of these features are 
common across major public sector projects and feature 
prominently in the NAO’s gold standard for project control 
in defence projects.? Where Gateway reviews identified 
shortcomings in the management of large scientific 
facilities, they were addressed by project teams. 


2.6 Concerns about the level of project management 
support available to teams have featured in Gateway 
reviews of 4th Generation Light Source, the HECToR 
computer, the research ship James Cook, the Institute 

for Animal Health and the Muon lonisation Cooling 
Experiment. Projects have sought to obtain project 
management skills from a variety of areas within Research 
Councils or Institutes, as in the case of the Muon lonisation 
Cooling Experiment; from direct recruitment of an 
experienced project manager as in the case of Halley VI; 
or from procuring external project management expertise. 
The RRS James Cook project, for example, following 

an abortive attempt at direct recruitment due to lack of 
suitably qualified candidates, procured a project manager 
from a consultancy company at a cost of approximately 
£1 million for the design and delivery phase. The research 
ship was delivered, within the original capital budget, by 
August 2006. This was three months after the timetable 
specified in the business case when the project was 
approved but by the contract delivery date. We recognise 
the value of high quality project management support to 
this and other projects, but believe that developing and 
sharing the collective project management skills of the 
Research Councils more effectively, drawing in external 
expertise as appropriate, could make this expertise 
available more readily and more economically. 


2.7 There was a good deal of stability of key personnel 
within the majority of teams taking forward projects - this 
applied to the Muon lonisation Cooling Experiment, ISIS, 
and to a large degree the Halley VI Antarctic research 
station and the Research Complex. However, projects 
such as the Institute for Animal Health had experienced 
substantial change to the composition of its project team, 
partly because it had identified the need for people with 
greater experience of managing and delivering large 
complex projects. Our interviews with project managers 
suggested they had close contact with colleagues 
operating similar facilities in the UK and overseas but 
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would welcome more frequent opportunities to discuss 
project management issues amongst colleagues in similar 
positions but working in different research fields. There 
may be a role for the new Science and Technology 
Facilities Council in facilitating access to, and the sharing 
of, project management experience. 


Defining target outcomes and outputs 


2.8 To be able to assess whether a project has achieved 
its objectives, business cases should include, as 
recommended by the Office of Government Commerce, 
success criteria covering the outcomes or outputs of 

the project. Where possible, criteria should be specific, 
measurable, achievable, realistic and timebound thus 
aiding project monitoring and evaluation. In May 2005, 
the Office of Science and Innovation established a 

new performance management framework for the 
science budget as a whole, covering the twin priorities 
of strengthening the science base and improving its 
exploitation to the benefit of the economy and public 
well-being. !° 


2.9 The majority of business cases we examined 
described success factors which could be used to help 
judge the worth and success of projects. The value of some 
of these factors was reduced as they were not specified in 
a way which would readily facilitate measurement. Where 
success factors were likely to be directly measurable 

they tended to focus on two areas. Firstly, measures of 

the satisfactory completion of the project, and secondly 
measures of the services or the capability that the new 
facility would provide, such as the number of people that 
could be accommodated on the new Halley VI Antarctic 
research station. Relatively few measures were proposed 
for either the extent of scientific activity undertaken on a 
new facility once it was operational or, the most difficult 
area to capture, the impact of that activity including 

the degree of exploitation, for example, by industry and 
public policy-makers. 


2.10 There were, however, examples of good and 
emerging practice from both current and previous 
projects (Figure 6) which could be more widely applied 
by project teams. Bibliometric measures — numbers of 
publications and citations — are widely used to measure 
research outcomes within the scientific community as a 
whole and can also be captured for science conducted on 
large facilities. The outcomes of peer review of research 
proposals can be used to judge quality of demand for 
large facilities. A consistent approach to capturing, 
recording and reporting such measures would help in 
judging the performance of large facilities, choosing 
proposals for new investments and undertaking any 
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evaluation of the programme of investments supported 
by the Large Facilities Capital Fund as a whole. Business 
cases for new facilities could incorporate the expected 
or target levels of such measures, as well as anticipated 
levels of demand, utilisation and technical performance. 


Delivering on budget - capital costs 


2.11 In preparing business cases and seeking project 
approval, most project teams revisited the capital cost 
estimates available at earmarking. For example, the 
Institute for Animal Health, Laboratory of Molecular 
Biology and Research Complex commissioned designs, 
or more detailed designs, of the buildings they required. 
These helped the project teams provide more robust cost 
estimates. The project teams also used benchmark data 
on the cost of building new laboratories to compare with 
the estimates they were proposing. Gateway reviews of 
the business cases of three further projects recommended 
action that teams could take to improve the completeness 
and robustness of costings, and this was acted upon. 


Current and pr vi s projects y Ic 


performance measures _ 


Research Complex. The project team proposed in its business 
plan a series of measures covering the number of high quality 
researchers (as judged by peer review) attracted to work at a 
facility, the demand for the facility compared to capacity and 
the number and impact of scientific outputs as measured by 
publication outcomes. 


HECToR. In response to comments received through the 
Gateway review of their business case, the team taking forward 
HECToR improved the range and specification of success 
factors. The factors now cover the project's contribution to 
scientific research, training and UK industry. The latter will be 
addressed by measures of the proportion of teams using the 
facility which include an industrial collaborator and a survey of 
users as a way of assessing the level of technology transfer and 
thus the impact of the facility. 


ISIS. The original ISIS neutron scattering facility has established 
and developed a range of performance measures since 
opening in 1985. These cover technical performance, demand 
for instrument time, machine availability, user satisfaction and 
publication outcomes. Examples of the levels and trends of 
some of these measures are illustrated in Appendix 2. 


Council for the Central Laboratory of the Research Councils. The 
Council has developed an outputs metric framework applying 
the new performance management framework developed 
by the Office of Science and Innovation (paragraph 2.8) to 
management of its existing large facilities and those that are 
currently being built. This covers, for example, assessments k 
of the quality of the research conducted by facility users, the 
international benchmark standing of its large facilities, facility 
utilisation rates and trends in peer review judgements on the 
research quality of applications for use of large facilities. 
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2.12 By autumn 2006, capital expenditure on the project team was forecasting that Diamond Phase | will 
ten projects underway was forecast to exceed their cost £263 million. This is four per cent above the original 
overall budgets at approval of £950 million by around budget, which did not include a contingency provision, 
£60 million (six per cent) (Figure 7). The only project set when Diamond was approved in 2001. The other 

to be completed to date — the RRS James Cook — was projects will not be ready until the end of 2007, at the 
delivered within budget in August 2006. Phase | of the earliest, and in some cases will not be operational until 
largest project - the Diamond Synchrotron - is due to 2009 or 2011, and thus they may well encounter further 
begin operations in early 2007. At autumn 2006, the — cost pressures. 


Capital budget Year capital Latest forecast Percentage Main reasons for change 
at approval budget as at change 
£ million approved autumn 2006 
£ million! 
Phase | 253.2 2001 263.2 4 Addition of contingency (+£10m) : 
_ Diamond A 
Phase II? 100.0 2001 120.0 20 Updating of base beamline estimate i 
(+£8m), additional equipment (+£3.5m), 
addition of test beamline (+£3.5m) and F 
addition of programme to improve : 
detectors (+£5m) ) 
"RRS James Cook 40.0 2002 40.0 0 f 
_ ISIS second target station 133.1 2003 145.6 9 Increase in the cost of the first suite of i 
__ with first suite of instruments instruments for the target station (+£12.5m) : 
: 4 
_ Energy Recovery Linac 12.9 2003 2 Nee 65 Addition of contingency (+£1m), i 
_ Prototype extension of period of project (+£2.5m), 
full costing of prototype (+£2m), extra : 
equipment components funded by North ; 
West Development Agency (+£2.9m) eI 
Halley VIS 34.0 2003 38.0 i2 Building contract tender higher than : 
expected (+£16m) partly offset by : 
subsequent efforts to reduce costs (£12m) _ 
_ HECToR 65.0 2004 59.4 -9 Cost reduction mainly due to project 
procuring hardware at a lower cost : 
than expected ' 
~ Muon lonisation Cooling 9.7 2004 9.7 0 ; 
_ Experiment Phase | 
Laboratory of 155.0 2005 164.0 6 Main cost increase has arisen from 
Molecule Biology inflationary pressures as project delivery 
has slipped since approval 
_ Institute for Animal Health 121.0 2006 121,0 
Research Complex 26.4 2006 26.4 0 
(excluding infrastructure) 
Total 950.3 1,008.6 6 


Source: National Audit Office analysis of Research Council data 
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2.13 Most projects had included contingency within the 
capital budget specified when the project was approved. 
For projects such as the Muon lonisation Cooling 
Experiment their contingency had been sufficient to cover 
cost pressures experienced by autumn 2006. However, 
the contingency allowed by five projects had not proved 
sufficient and thus teams were forecasting that their 

costs would exceed the capital budget set at approval 
(see Figure 7). There have been four main causes of 

these pressures: 


a __ Use of early incomplete estimates has created 
costs pressures of approximately £20 million. 
When the ISIS second target station was approved 
in 2003 it was envisaged that £27.5 million would 
be needed to fund seven new instruments. As more 
detailed plans for the instruments were worked up 
with user groups, Cost estimates were revised to 
£40 million. Some £10 million of the extra funding 
to meet the higher costs will come from international 
sources. For the Energy Recovery Linac Prototype, 
costings were produced quickly in response to the 
option of establishing a prototype rather than moving 
ahead immediately with a full 4th Generation Light 
Source. As a consequence, the initial estimate of 
£12.9 million for the original concept proved too 
low, and £2 million was added to the project's cost 
despite the team containing costs by, for example, 
using concrete shielding already on site and re-using 
equipment from other laboratories. A contingency 
of £1 million has also been added to the Energy 
Recovery Linac Prototype budget since the project 
was approved. 


b Optimistic assumptions of project timing and 
duration have resulted in cost pressures of around 
£10 million. For example, the redevelopment of 
Laboratory of Molecular Biology has been delayed 
by around two years, partly because the developer of 
the wider site where the Laboratory is to be located 
has not secured planning permission as quickly as 
expected. The delay has created inflationary pressure 
contributing much of the £9 million increase in the 
cost of the project as forecast at autumn 2006. 


c Enhancements made to projects after approval 
have added £15 million to costs. For example, 
on Diamond Phase II, enhancements costing 
£12 million are planned. These include a test 
beamline and a programme to improve the detectors 
which capture data from beam line experiments. 
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d _—_— Higher than expected tender prices. In the case 
of Halley VI the tender price for building the new 
Antarctic research station was £16 million (or 
75 per cent) higher than expected and could not be 
covered by the project's budget. The project team 
responded by revisiting the phasing and scale of 
the project and identified and costed a wider range 
of options for delivering the science objectives. 
The team has contained the level of cost growth of 
the overall project to £4 million by cutting the size 
of the new facility and by combining construction 
with demolition of the existing Halley V station. As 
a result of combining the two phases some science 
will be suspended whilst Halley VI is built. 


Delivering on budget — costs in use 


2.14 The eventual cost of operating some new facilities 

is likely to exceed the initial capital cost of a facility, for 
example, the costs of operating the Diamond Synchrotron, 
which is likely to cost around £380 million to build 
Phases | and II, are predicted to be in the region of 

£32 million per annum when the first seven beamlines are 
operational which should be in 2007. When the second 
set of beamlines become operational in 2011, the project 
team predicts that the cost of operating the facility, which 
should still have an operating life of over 20 years, will 
rise to around £46 million per annum. When operational, 
the host Research Councils will have to meet the staff, 
electricity and other direct operating costs of large 
facilities from their annual budgets. They will also incur 
annual depreciation charges which are in addition to an 
annual charge for the cost of capital, which is incurred as 
soon as funds are tied up in the capital facility. 


2.15 More detail could have been included in business 
cases to aid the assessment of affordability. For example, 
the original business cases prepared for the Institute for 
Animal Health and the Halley VI Antarctic research station 
did not provide estimates of their operating costs. The ISIS 
business case provided a figure for operating costs but 
did not set out what cost elements had been included or 
key assumptions such as the number of operating days. 
Given the difficulties of predicting future operating costs, 
teams could have done more to explain the main drivers 
and uncertainties in their estimates and give a feel for the 
likely range of costs, by drawing on sensitivity analysis, 
rather than specifying a single figure. 


2.16 By autumn 2006, five of the six most mature projects 
had revisited their approved business case estimates of 
annual operating costs and were forecasting significantly 
increased operating costs (Figure 8). The most significant 
impact will be on the Council for the Central Laboratory 
of the Research Councils which is hosting both the 
Diamond Synchrotron and the second ISIS target station. 
The anticipated total increase in its operating costs is in 
the region of £25 million per annum at 2006-07 prices 
or around 12 per cent of the Council’s current annual 
operating expenditure. If the Council does not secure 
additional resources, this degree of cost growth could 
exacerbate existing constraints which, for example, limit 
the number of days the Council operates the existing ISIS 
target station to 180 days. This is some 40 days less than 
the 220 days that could be accommodated within the 
timetable required for maintenance and other work. 


2.17 Three main factors have pushed up the operating 
costs of these five projects: 


m Some cost elements have increased at unexpectedly 
high rates. Each of the projects will be adversely 
affected if the rise in fuel and electricity prices that 
has occurred since their approval is sustained. 


mw Benchmarks proved difficult to apply to the 
Diamond Synchrotron project. Original estimates 
of the running costs for Diamond drew on UK 


Expected life 


years 
Diamond Phase | and II 25 
RRS James Cook 25 
_ ISIS second target station? 1 
© HECToR? 6 
~ Muon lonisation Cooling Experiment 3 


| Source: National Audit Office analysis of Research Council data 
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experience of running the existing synchrotron at 
Daresbury and knowledge of other synchrotron 
sources. However, the project team has found that 
inadequate allowance was made for the much larger 
Diamond infrastructure and its consequent impact 
on power consumption, staff requirements and 
premises costs. 


m The HECToR team’s experience indicates that it can 
be difficult to predict the price of IT support services 
at business case. 


2.18 The opening of large facilities will increase the 
opportunities available to scientists to undertake research 
in these areas. Many of the users of these new facilities 
will be funded through grants provided by Research 
Councils. For these facilities to be fully utilised, scientists 
will need to compete successfully for research funding, 

by peer review, against other calls on limited Research 
Council budgets. Our review suggested more work is 
needed by Research Councils to examine the potential 
impact of these facilities on the future demand for research 
funding, their capacity to support any new demand, or the 
effect of doing so on other areas of research activity. As 
the new facilities come on stream, the Research Councils 
should monitor the impact on the demand for research 
funding and ensure lessons are learned for the appraisal of 
future proposed facilities. 


FEY) esincred annual operating costs of some projects have increased significantly! 


Estimate in approved Latest estimate at Percentage 
business case autumn 2006 change 
£ million per annum £ million per annum 
24.4 46.1 89 
2.8 3:5 25 
5.4 99 83 
re 8.2 52 
1.6 1.8 12 


1 All estimates are at 2006-07 prices and exclude depreciation and capital charges. It was not possible to compare costs estimates for the other five 
projects we visited. By autumn 2006, two projects - the Research Complex and Institute for Animal Health — had not revisited estimates made when the 

projects were approved early in 2006, although the Institute was in the process of doing so. Neither the Laboratory of Molecular Biology nor Halley VI had 
prepared a full estimate of annual operating costs at the time the projects were approved. The capital budget for Energy Recovery Linac Prototype is intended 
to cover the cost of research and development of the Prototype. As at autumn 2006 there was no approved programme for exploiting the Prototype once it 
has been developed and thus there were no associated operating costs for the project. 


2 Figures ar 
by discontinuing the operation of two instruments on the existing station. 


re for the cost of operating a full suite of 18 instruments on the second target station. The latest estimate reflects planned savings to be achieved 


3. As at autumn 2006, HECToR had not finalised contracts for service provision so the latest estimate is based on updated predicted prices. 
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2.19 Opportunities to trade off capital costs against 
operating costs were rarely extensively explored. An 
exception is the Laboratory of Molecular Biology. This 
considered a range of chilling plant options to give the 
best lifetime cost solution and the Laboratory will include 
interstitial floors. These provide separate space to assist, 
and thus minimise the impact and costs, of maintenance 
and reconfiguration of the new laboratory. The project 
team also recognised that introducing robotics could 
reduce the lifetime costs of sample handling and thus the 
building has been designed to allow for this. 


Delivering on time 


2.20 Figure 9 shows that progress to date against delivery 
dates specified when projects were approved has been 
mixed. Good progress has been made with Diamond 

and RRS James Cook — the two most mature projects. 

As mentioned in paragraph 2.6 RRS James Cook was 
delivered in August 2006 and, despite missing a number 
of milestones, Diamond is forecast to commence 
operations in January 2007. Although this is four months 
after the original date set in 2001, it is the date specified 
when Diamond Light Source was created and took 


responsibility for building and operating the synchrotron 
in 2002. In contrast, three other teams — Laboratory of 
Molecular Biology, Halley VI and the Energy Recovery 
Linac Prototype — are now predicting delivery dates which 
are a year or more later than the approved date. Further 
risks remain to the completion of these projects and the 
other undelivered projects, two of which are still at an 
early stage. 


2.21 The time required to undertake the procurement 
process and let the contracts for the Halley project was 
longer than had been allowed for in the approved business 
case. This has pushed back the starting date for operations 
on the new Halley VI research station by a year, although the 
project team reported that the forecast delivery date for the 
overall project which includes decommissioning the existing 
research station Halley V remained unchanged (Antarctic 
summer 2009-10). The Energy Recovery Linac Prototype 
project has also been extended mainly because the team 
decided more time was needed for design and testing of 
what is a prototype for a larger potential project. The project 
has also suffered further slippage due to delays in the 
delivery of key components. The Muon lonisation Cooling 
Experiment encountered problems in getting adequate staff 
to take forward initial project work. 


Research Complex | 

Institute for Animal Health main laboratory building | 
Laboratory of Molecular Biology | 

i Muon lonisation Cooling Experiment | 
: HECTOR | 
Halley VI! | 

Energy Recovery Linac Prototype | 

ISIS second target station with first suite of instruments | 
RRS James Cook? | 


Diamond Phase |? 


D 
| 
| 
| 
| 


ate of approval 


Planned date of project completion at approval 
Autumn 2006 


Latest forecast of 
completion date 
at autumn 2006 


| 


| i] 


| 
| 
| 
| 
| 


| | | 
April April April April April 
2001 


April April April April April April April 


2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 


g MB Planned period for completing the project {ff Movement from planned completion date 


4 
4 
4 Source: National Audit Office analysis of Research Council UK data 
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2.22 Project slippage can increase the pressure on cost 
budgets. For example, Research Council staff may charge 
their time to a project over a longer period and project 
teams may need to give new suppliers shorter periods 

to deliver equipment and services and this may push up 
prices. Ultimately, slippage will delay the benefits that will 
flow from new facilities and could thus reduce their value. 
For most of the projects we visited, however, there is little 
evidence that the degree of slippage experienced to date 
will significantly threaten their longer term contribution 

to science. 


2.23 Further delays in projects, however, could be 
problematic. For the Halley project, any significant 
slippage means a year’s delay (because of weather 
constraints). This would increase the risk that the project 
would not be completed before any breaking and 
separation of the ice-shelf where the current Halley station 
is located. For the Laboratory of Molecular Biology, further 
delay beyond 2011 could have a detrimental impact 

on the recruitment and retention of staff and thus the 
institute’s international scientific position. Whilst if phases 
| and II of the Muon lonisation Cooling Experiment are not 
completed until 2010 it could significantly compromise the 


_ Feature Diamond RRS James ISIS 

Phase | Cook second 
and Il target 

station 

_ Transfer of design risk Low High Low 

"Level of competition for Strong Strong Strong 

_ main procurement/s 

Level of competition Mixed All in Mixed 

for direct component main 

| procurements contract 

: Pain/gain share Some No No 

» Significant delay in Yes No Yes 


"contract milestones in the 
period from contract 
_ award to autumn 2006 


Source: National Audit Office 
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experiment’s prime objective of informing the international 
design of a “factory” capable of producing a very intense 
and focused beams of neutrinos. Consequently, the 
project team has sought to tackle risks to those parts of the 
project being undertaken overseas, as well as those being 
undertaken in the UK. For example, in early 2006 the 
team provided around £60,000 to help overseas partners 
to purchase equipment essential for the progress of the 
experiment. In return, the partners have agreed to take 

on increased responsibility for other equipment that the 
project will require at a later date. The team recognises, 
therefore, that in assisting international partners it must 
strike a balance between ensuring good project progress 
and maintaining strong incentives for international partners 
to deliver their planned contributions. 


Procurement strategies 


2.24 Eight of the ten projects we examined have designed 
their procurement strategy and the position for each is 
characterised in Figure 10. Contracts were open to UK 

and overseas companies with, for example, a Norwegian 
company winning the contract to build the RRS James Cook. 


tion of project procurement strategies and outcomes! 


Energy Halley VI HECToR? Muon Institute for 
Recovery lonisation Animal 
Linac Cooling Health main 
Prototype Experiment building 
programme 
Low Medium High Low Medium 
No main Weak Mixed No main Mixed 
contract contract 
Mixed All in All in Mixed All in 
main main main 
contract contracts contract 
No Yes Subject to No Subject to 
negotiation negotiation 
Yes Yes Yes No 


rred bidders had been identified but contracts had not been awarded for the main components of HECToR. 


s Laboratory of Molecular Biology and the Research Complex as they had not prepared a procurement strategy which had passed 
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2.25 Where design risk has been retained by the Research 
Council this has normally been because of a belief that 
the skills to address technical design issues are more 
readily available to the Research Council than prospective 
contractors, and/or a desire to retain and develop in-house 
technical skills and knowledge for later operation of the 
facility. In general, more design risk is transferred for 

the building construction element of each facility than 

the machine components (where applicable) which are 
generally procured and installed directly by the Research 
Council in line with their own design. Where design risk 
has been transferred it has been in a proportionate rather 
than wholesale fashion with, for example, the RRS James 
Cook project using a Statement of Requirements supported 
by quality and outfitting standards, and a list of accredited 
suppliers for key components. 


2.26 The degree of competition for the main contracts 
was variable. There was strong competition for the main 
buildings for both Diamond and ISIS, and for the research 
ship James Cook. For Halley VI, competition was strong 
for design but only one compliant company expressed an 
interest in constructing the new Antarctic research station. 
For HECToR, competition for the hardware component 
was strong but competition for computational support and 
engineering, and the provision of facilities management, 
was weak. The Halley VI project had conducted some 
market engagement prior to finalising its procurement 
strategy. HECToR had engaged in extensive market 
engagement which had indicated suppliers in each sector 
(hardware, computational science and engineering, 
facilities management) favoured being appointed as 

leads for the overall procurement rather than bidding for 
components separately though the latter approach was 
acceptable for most. In the event separate tenders for each 
component were invited, albeit with the possibility of 

the facilities management provider taking on the winning 
hardware bidder's contract. 
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2.27 Levels of competition for direct procurement of 
specific components were mixed. In some cases a limited 
response was because the number of potential suppliers of 
specialised equipment was limited. In others, companies’ 
perceptions of the technical nature of the facility, rather 
than the demands of a particular work package, was 

a more likely explanation. The Diamond Synchrotron 
project, for example, received only three compliant 

bids to construct and supply services to four beamline 
experimental cabins. This was despite the fact that the 
work was not especially technically demanding and that 
96 companies had expressed an interest in some or all of 
the work when it was first announced by Diamond. 


2.28 Provisions for liquidated damages for the impact 
of delivery delays is a widely used form of incentive 

for large facilities projects. They are incorporated in 
many of the Diamond Synchrotron project’s contracts 
for machine component procurements, and on the 
main building delivery dates. The project also sought 

to introduce additional damages on the main building 
contract for missing access dates for machine installation 
after the specification for the building had been issued 
and tenders had been received and evaluated. The 
successful contractor proposed a bonus for meeting access 
dates or a contract cost increase and the matter was not 
pursued. Delays in gaining access to the new building 
by the due dates, to allow installation of machinery, has 
proved problematic and has meant machine installation 
progressing alongside building works for longer than 
intended. Practice on liquidated damages varies on an 
item by item basis elsewhere, balancing the likely price 
premium against the impact of delivery slippage. No 
delivery penalties were included in contracts for the 
supply of components for the Energy Recovery Linac 
Prototype, for example, and there have been significant 
delays in some components. The contract for Halley VI 
incorporates pain/gain share provisions. 


2.29 Figure 11 summarises some of the lessons to be 
learned from the early projects. 


>m lessons of — 


Deeper analysis of design risks to separate: 


those which can and should be transferred to 
a contractor; and 


m those which could be transferred but are likely to: 
— lead to a disproportionate price premium; 
— deter bidders altogether; 
— cause the contractor to fail; or 


— transfer valuable technical learning which is needed 
for facility operation. 


Greater consideration of the incentives for delivery dates of 
key components or milestones and use of pain/gain sharing. 


Identification of work with a limited pool of suppliers and 
work with less demanding requirements. This will allow 
packaging of work components to maximise potential 
market interest across the project’s work content. Such 
packaging should have regard to early market engagement 
which would indicate whether specialist suppliers will 

be reluctant to bid unless work within their specialism is 
packaged with other components. 


More active promotion and market engagement for those 
work packages where perceptions of the scientific nature of 
the overall project carry a risk of deterring capable bidders 
unfamiliar with work on such facilities. 


Source: National Audit Office 
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Planning for operations 


2.30 Two projects are currently approaching the 
operational phase — the Diamond Synchrotron and the 
research ship James Cook. The Diamond Synchrotron 
project has established a user office and procedures for 
users to access the facility. The first call for academic 
proposals, covering the period January to September 2007, 
was issued in September 2006, with a deadline for 
submissions of 1 November 2006. The project has also 
established the Diamond Industrial Science Committee 
(DISCo) to advise the project on opportunities for industry 
to be engaged in research at Diamond, and the best 
means for promoting such opportunities. 


2.31 The Natural Environment Research Council has 
designed the scientific cruise programme for RRS James 
Cook, which begins with the first science cruise in 

March 2007. As at autumn 2006, the existing RRS Charles 
Darwin crew had been transferred to the new vessel 

and were engaged in commissioning trials, training 

and familiarisation. Publication outcomes from Natural 
Environment Research Council research vessel cruises are 
not tracked and this will need to be addressed for the cruise 
programme as a whole, including the RRS James Cook. 


2.32 Both projects have sought to generate public interest 
in the projects through press releases, and the Diamond 
Synchrotron project has its own communications team 
for this purpose. At the time of our audit there were no 
detailed plans yet in place for educational programmes 

or materials associated with the facilities, despite their 
potential interest to the public at large. 
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Review of existing literature 


1‘ The study team reviewed English and French 
language literature on existing large scientific facilities 
around the world, using internet search engines, the 
electronic catalogues of the London School of Economics 
and the Wellcome Trust, and the online enquiry systems 
of scientific journal publishers. The literature review 
covered evaluative studies of decision making processes, 
and the scientific and wider economic impacts of public 
investment in research in general and large scientific 
facilities in particular. The purpose was to both test our 
findings against comparable evaluative work and learn 
how UK processes compare with others. 


2 The study team reviewed previous NAO reports 

on groups of large capital projects, including the annual 
major defence projects report for lessons on methodology 
and general programme and project management. The 
team also drew on the NAO’s gold standard for project 
control in defence projects. This is part of the NAO’s 
on-going programme of work to better understand what 
drives the performance of major defence projects. A 
summary of the gold standard is at www.naodefencevim. 
org/downloads/pdf/gold_standard_poster.pdf. 


Review of road map and prioritisation 
documentation 


3. The study team reviewed documentation associated 
with the assembly of the large facilities road map, the 
prioritisation of projects within the road map, and the 
earmarking of funds to prioritised projects. This gave the 
team an understanding of the current form of the system 
and how it has developed since introduction, and allowed 
us to identify and review the content of key elements such 
as initial project proposals. 


26 BIG SCIENCE: PUBLIC INVESTMENT IN LARGE SCIENTIFIC FACILITIES 


Study methods 


Project visits 


4 The study team reviewed in detail the evolution, 
status and plans of all ten projects prioritised for support 
from the Large Facilities Capital Fund prior to the 2005 
road map.'! The purpose was to ensure that findings were 
based on a wide view of the nature of projects and the 
issues experienced, including the evidence supporting 
their bid for prioritisation. It was considered especially 
important to cover all ten projects given their very 
different nature, and the consequent risks of relying on a 
small sample. 


5 The study team visited the site of each project for 
three days and conducted semi-structured interviews with 
key personnel. These included but were not limited to the 
project sponsor, the project manager and relevant finance 
staff. Interviews covered the progress of the project 
to-date; project management arrangements; the chosen 
procurement strategy and where relevant, any changes to 
the agreed cost, specification or delivery since the project 
was agreed. Project visits also included review of project 
documentation, a list of documentation requirements 
sent well in advance to project managers, and a standard 
feedback form provided to each project manager three to 
four weeks after each visit. 


Semi-structured interviews and 
discussions with Research Councils and 
the Office of Science and Innovation 


6 ~The study team interviewed research directors of 
six Research Councils representing both providers and 
users of large scientific facilities (PPARC, MRC, CCLRC, 
BBSRC, NERC and EPSRC). These interviews were on 

a semi-structured basis and sought to capture views on 
the strengths and weaknesses of the current prioritisation 
and funding system and opportunities for improvement. 
A similar interview was conducted with the Assistant 
Director responsible for the Large Facilities Capital Fund 
at the Office of Science and Innovation. Preliminary 
conclusions were presented verbally to, and discussed 
with, finance directors of Research Councils. The purpose 
of these discussions was to test emerging conclusions and 
recommendations with those having most knowledge of 
current systems and procedures. 


= In addition, an early meeting with the secretariat 
of Research Councils UK was followed up by further 
discussions to clarify the nature and practical application 
of road map and prioritisation processes. 


Case studies of previous 
public investments 


8 Two previous investments, the research ship Charles 
Darwin and the first ISIS target station, were reviewed 

to identify lessons in performance measurement and 
monitoring which might be applied to new facilities, 

or inform judgements on proposals for such facilities. 

The ISIS project in particular demonstrated a wide set of 
metrics which are illustrated in Appendix 2 of the report. 


Interviews with senior scientists in 
technology-based companies 


9 Semi-structured interviews were held with senior 
scientists in BAE Systems, BT, Rolls Royce and Unilever. 
The purpose was to seek parallels with public sector 
decision-making processes on large technical facility 
investments. In practice the investments being made 
were of a different scale and nature to the public sector 
and assessment criteria were aligned with commercial 
business unit strategies which had limited parallel in the 
public sector. 


APPENDIX ONE 


Questionnaire to scientific stakeholders 


10 The study team identified a group of individuals 
whose roles as subject group or thematic chairmen, vice 
presidents, or equivalent within scientific and engineering 
societies and academies, or as chief scientists within 
technology-based companies, made them likely to be 
aware of issues surrounding access to large facilities for 
scientists within their communities of interest. This group 
included — but was not entirely made up of — users and 
potential users of existing facilities from academic science 
and industry. A self-completion questionnaire was used to 
capture their views of current and future investment in UK 
large facilities. 


11. From a target group of 130 individuals, 62 
completed questionnaires were returned. 


Use of an expert panel 


12 In order to provide guidance on the design and 
preliminary conclusions of the study, an expert panel 

was recruited. It was made up of an overseas member 
engaged in facility planning at European level, an expert 
in science policy analysis, and an expert in managing the 
development and operation of a large nuclear facility. All 
three had significant experience as users of large facilities 
within the physical sciences. A further two members were 
recruited on the basis of their knowledge and experience 
of life and earth sciences facility user communities. 


Professor Sir Chris Llewellyn-Smith 
Director, United Kingdom Atomic Energy Authority, 
Culham 


Dr Jorgen Kjems 

Director, Riso National Laboratory, Denmark 
and member of the European Strategic Forum 
on Research Infrastructure 


Professor Ben Martin 
Science Policy Research Unit (SPRU) — Science and 
Technology Policy Research, University of Sussex 


Professor Duncan Wingham 
Centre for Polar Observation and Modelling, 
University College London 


Professor David Stuart 
Professor of Structural Biology, Wellcome Trust Centre for 
Human Genomics, Oxford 


The panel was consulted before beginning the fieldwork 
and again when developing the initial findings in order to 
ensure that key issues were fully and properly considered 
when designing the study and drawing conclusions. 
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APPENDIX TWO 


As mentioned in paragraph 2.10 and Figure 6, project 
teams introducing and managing new facilities might 
usefully draw on the performance measurement practices 
used at the existing ISIS facility. 


The ISIS facility 


The ISIS neutron scattering facility opened at the 
Rutherford Appleton Laboratory, Harwell in Oxfordshire 
in 1985. Neutrons are produced by the spallation process. 
A heavy metal target is bombarded with pulses of highly 
energetic protons from a powerful accelerator, driving 
neutrons from the nuclei of the target atoms. There are 18 
neutron beam channels which feed different instruments. 
These include, for example, diffractometers which look 

at structures — that is the spatial distribution of atoms, 
molecules, or larger scale structures. 
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Performance measurement 
case study — the ISIS 
neutron source 


Performance measurement at ISIS 


Since the opening of ISIS, measures covering demand, 
availability, and technical performance have been 
monitored. Annual surveys of users have also been 


“conducted to monitor satisfaction levels with different 


aspects of service. 


The facility maintains records of all publications 
associated with scientists’ use of the ISIS facility. 
Identifying these relies on accurate reporting of 
publication outcomes by facility users. Citation levels of 
these publications have been analysed on occasion for 
specific reports but are not routinely monitored. 


Levels and trends of some of these measures over the 
lifetime of the facility are illustrated below. 
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Machine availability 
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Source: National Audit Office and the Council for the Central Laboratory of the Research Councils 
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_ Diamond Synchrotron — Phases | and II 


_ Lead Research Council 


~ Council for the Central Laboratory of the Research Councils (CCLRC) 


: Location 


_ Harwell Science and Innovation Campus, Oxfordshire 


_ Facility description 
_ Diamond is a third generation synchrotron radiation source. The 
_ facility is based on an accelerator for electrons, a storage ring, 


__and 24 cells with space for up to 40 X-ray beamlines directed 
__ at samples. These will enable investigation of materials for the 


physical, life and environmental sciences. Diamond is being 


_ delivered and run by Diamond Light Source Ltd, a joint venture 

_ company with Wellcome Trust which is an independent charity 

_ which funds research aiming to improve human and animal 
health. Wellcome Trust holds 14 per cent of the shares in Diamond 
_ Light Source and the Council for the Central Laboratory of the 

_ Research Councils holds the remaining 86 per cent. Diamond will 
_ replace the synchrotron radiation source at CCLRC’s Daresbury 

_ laboratory in Cheshire. 


: Phasing 


' m Phase | — building, accelerator, storage ring and 


seven beamlines 


mPhase Il - installation of a further fourteen beamlines and one 


test beamline 


: m Phase Ill — installation of a further ten beamlines 


i Status at autumn 2006 


_ The project passed Gateway 3 for the main building contract in 
_ August 2003. The building housing the administrative office was 
_ completed in January 2005. Electrons were first circulated in the 
_ Booster in December 2005. Stored beam was achieved in the 


Storage Ring in May 2006 which allowed the first observation 


__ of synchrotron light. The synchrotron building completion was 
~ achieved in August 2006 and beamline commissioning began in 
_ October 2006. Gateway 4 is scheduled for January 2007. 


: Capital cost and funding (Phases | and II) 


Phase | Phase II 

E Capital cost £ million £ million 

_ Budget at project approval 253.2 100.0 
- in 2001 

Latest estimate at autumn 2006 263e2 120.0 

_ Variation +10.0 +20.0 

(3.9 per cent) (20 per cent) 

Reasons for variation Addition of Updating of base 

contingency beamline estimate 

provision  (+£8m), additional 


equipment (+£3.5m), 
addition of test 
beamline (+£3.5m) 
and addition of 
programme to 
improve detectors 


(+£5m) 


At project approval 
in 2001 


When Diamond Light 
Source took responsibility 
for building the 


January 2007 


synchrotron 

Latest estimate at January 2007 
autumn 2006 

Variation since +4 months 


approval in 2001 


Reasons for variation Setting up of 


joint venture and 
start of enabling 


works 


Operating costs (Phases | and Il) 


At project approval in 2001 
Latest estimate at autumn 2006 


Variation 


Reasons for variation — original estimates based on operation of 


September 2006 


£24.4 million per annum for 
Phases | and II 


£46.1 million per annum for 
Phases | and |I 


+£21.7 million per annum : 
(88.9 per cent) 
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Phase | Phase Il | 
Sources of funding £ million £ million 
Larges Facilities Capital Fund ~ 82.4 k 
Office of Science and Innovation 226.2 - 
Wellcome Trust 36.8 16.8 
CCLRC - 70 
Other Research Councils ~ 14.0 
Delivery timetable (Phases | and II) 

Phase | Phase II 


1 beamline per 
quarter from 

quarter 1 2007 

to quarter 3 

2010 (15 in total) © 


as above 


1 beamline per 
quarter from 
quarter 2 2007 
to quarter | § 
2011 (15 in total) | 
Start +3 months 

End +6 months 


Later Phase | 
start and the 
addition of a test 
beamline 


Daresbury synchrotron — a much smaller machine. Also increases 


in unit energy costs, business rates and commercial insurance. 
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(2200 beamline days per year) and associated beamline 


Reasons for variance — increased forecasts of user demand and ‘ ; 
requirements (12) were included. 


technical capability to achieve greater beam brightness. There 
had been no major changes to specification since the project was 
approved in 2001. 
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Diamond Synchrotron — Phases | and II continued 
Specification Critical Success Factors y 
Electron Storage Number — Horizontal m focus on science 
Energy Ring of Cells — Emittance ae : 
Circumference (smaller = et ene es : 
Giga brighter) technical management ke 
Electron Nm rad , ; Es 
Volts Meroe recruitment, retention and development of expertise . 
Original (1993 3 301 16 30.0 im repuichon j 
Woolfson Report) No metrics were associated with these critical success factors : 
Current 3 562 24 Zep in the business case, though projections of life science demand : 


Summary of Project-Specific Strengths 


mm Strong commitment to project at all levels (paragraph 1.11) 
Procurement strategy 
m Drive and focus of dedicated delivery body - Diamond Light 
m Design contract for building, in-house design of machine Source Ltd (paragraphs 2.5 and 2.20) 
and instruments 


@ Strong CCLRC, Wellcome Trust and external advisory team 
m Contracts for building project management, quantity surveying (paragraph 2.5) 


and planning supervision 


m@ Main contract for building construction including mechanical Summary of Project-Specific Issues 


and electrical services — two-stage tender with main contractor m Estimation of operating costs (paragraph 2.16) 
for stage one (assembly of package bids) awarded stage two pons, 
contract (delivery) once work packages over 80 per cent of i Delays in building handover (paragraph 2.28) 


total value had been determined m Packaging and marketing of directly procured non-specialist 


m Separate direct procurements for supply of machine and work (paragraph 2.27) 


isto ene componenls @ Handling of design and integration (building — machine 


mm In-house assembly and installation of machine and — beamlines) risk (paragraph 2.24 and Figure 10) 
beamline components m Contract incentivisation (paragraph 2.28) 

m@ Contingency provision in Phase | capital costs and incomplete 
capital cost estimates for Phase II (paragraph 2.13 and 
Figure 7) 


m Weakness of critical success factors (paragraph 2.9) 
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Royal Research Ship James Cook 


Lead Research Council 


Natural Environment Research Council (NERC) 


Location 


Worldwide — primarily Atlantic cruise programme 


Facility description 


The research ship RRS James Cook is the successor to the RRS 
Charles Darwin, part of NERC’s fleet of two such research vessels. 
It will be used for a programme of scientific cruises to conduct 
oceanographic and marine studies and is equipped to launch and 
recover heavy marine equipment such as submersible or towable 
sensing and monitoring devices. It also has on-board laboratory 
space and data analysis facilities. 


Phasing 


The ship has been designed and built under a single contract. 


Status at autumn 2006 


The ship was delivered to Southampton and handed over to 
NERC on 31 August 2006. She was undergoing commissioning 
trials in autumn 2006. 


Capital cost and funding 


Capital cost £ million 
Budget at project approval in 2002 40 
Latest estimate at autumn 2006 40 
Variation none 
Sources of funding £ million 
Large Facilities Capital Fund 25 
NERC 1S 


Delivery timetable 


At project approval in 2002 May 2006 
At contract award in 2004 August 2006 
Actual delivery August 2006 
Variation since approval in 2002 +3 months 


Reasons for variation — one month extended bidding time 
following feedback from suppliers. Two month extension to allow 
two year build time following receipt of tenders. 
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Operating costs £ million per annum 


At project approval in 2002 228 
Latest estimate at autumn 2006 6)55) 
Variation +0.7 


(+25.0 per cent) 


Reasons for variation — increased marine diesel price, increased 
fuel requirement due to larger vessel and dynamic positioning, 
greater maintenance and spares requirement for larger vessel. 


Specification 


Same as business case approved in 2002: endurance (50 days), 
maximum speed (15 knots), dynamic positioning capability, 
marine complement (22 single berths), scientific complement (32 
single berths), payload, stern gantry (30 tonnes — 8 metre lift), aft 
port and starboard knuckle cranes (40 tonnes.m). Minor changes 
to laboratory areas (from 285m? to 278m’). 


Changed from business case approved in 2002: one extra deck 
container, improved network capability (Cat 6 to Cat 7), addition 
of enhanced communications, provision of £300,000 of spares. 


Critical Success Factors 


Success factors were developed after the business case was 
submitted for Gateway review. The factors included that the vessel 
needed to be fit for purpose, delivered on time and within budget. 


Procurement strategy 


Tender for design and build based on Statement of Requirements 
(SoR) and Quality and Outfitting Standards (QOS). A range of 
acceptable suppliers for some components was specified and 

the contractor had to evidence the viability of any alternative 
suppliers he wished to use. Beyond this the design of the vessel 
was a matter for the contractor, subject to submission of drawings 
to the project team and its advisors, and on-site checking of the 
conformity of installations with performance requirements. 


Summary of Project-Specific Strengths 


= Compact but appropriately skilled project team with user 
representation (paragraph 2.5) 


mw Extensive consultation to establish stable statement of 
requirements (paragraph 2.25) 


m Effective transfer of design risk (paragraph 2.25) 
Delivery achieved within budget and within a build period 
consciously extended by three months (paragraph 2.6, 
Figures 7 and 9) 

Summary of Project-Specific Issues 

m Cost of project management (paragraph 2.6) 


m Estimation of operating costs (Figure 8) 
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ISIS Neutron Source Second Target Station — Phase 1 


Lead Research Council 


Council for the Central Laboratory of the Research Councils (CCLRC) 


Location 


Harwell Science and Innovation Campus, Oxfordshire 


Facility description 


ISIS is a pulsed neutron source. A heavy metal target is 
bombarded with pulses of highly energetic protons from a 
powerful accelerator, driving neutrons from the nuclei of the target 
atoms. Instruments allow study of the interaction of the resulting 
neutron beams with samples. The second target station will 
accommodate up to 18 instruments. Target station and instruments 
will be optimised for study of soft condensed matter, bio-materials, 
advanced materials and nanotechnology. 


Phasing 


_ & Phase 1 - target station and seven instruments 


i m Phase 2 - a further eleven instruments 


; Status at autumn 2006 (Phase 1) 


STAT AS SERIE LEIP EL LE ETS BEREAN LES DS BEET IIT 


Phase 1 of the project passed Gateway 3 (contract award) in 
May 2006. The building housing the new target station was also 
completed later in 2006. 


Capital cost and funding (Phase 1) 


Capital cost £ million 
Budget at project approval in 2003 SIS | 
Latest estimate at autumn 2006 145.6 
Variation +12.5 

(+9.4 per cent) 


Reasons for variation — as more detailed plans for the first suite of 
seven instruments were worked up with user groups, cost estimates 
were revised up from £27.5 million to £40 million. Funding for 
£10 million of the cost growth will come from international sources. 


Sources of funding £ million 
Large Facilities Capital Fund 127.9 
1) CCLRC LG 
EU Framework 6 7.0 
; Bilateral agreement with Spain 320 


_ Delivery timetable (Phase 1) 


_ At project approval in 2003 


34 


September 2008 


Latest estimate at autumn 2006 October 2008 
Variation +1 month 
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Operating costs (for both Phase 1 and 2) 


£ million per annum 


At project approval in 2003 


54 
Latest estimate at autumn 2006 9.9 
Variation +4.5 


(+83.3 per cent) 


Reasons for variation — estimate available at project approval 

did not specify in detail what cost elements had been included. 
However, project affected by higher than expected energy 

costs. Latest estimate reflects planned savings to be achieved by 
discontinuing the operation of two instruments on the existing station. 


Specification (Phase 1) 


Original 2002 Business Case specified seven first phase 
instruments with five of these transferred from current 
target station. Initial average current 60 microamp. By 
project approval in 2003, it had been decided that the 
seven new instruments should be provided in the 


first phase. 


Current — Seven first phase instruments but with the 
discontinuation of two instruments on the existing 
station to reduce overall running costs. Initial average 
current 30 microamps rising to 60 microamps 


after four months. 


Procurement strategy 


m Design contract for building, in-house design of machine 
and instruments 


@ Main contract for building construction including mechanica 
and electrical services 


m Separate contracts for supply of steel for building and 
target monolith 


mw Separate direct procurements for supply of machine and 
instrument components 


m In-house assembly and installation of machine and 
beamline components 
Critical Success Factors 


m@ Delivery of the capital facility — on time, to budget, capability 
met, instruments meet scientific aspiration, minimise negative 
impact on ISIS 1 operations 


m Level of funding from overseas 


m Use and impact of facility - comparison of scientific output 
(number of publications and impact) with other facilities, 
stakeholder involvement 


m= Mojor skills retained and developed 
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_ ISIS Neutron Source Second Target Station - Phase 1 continued 


_ Summary of Project-Specific Strengths Summary of Project-Specific Issues 
m Experienced and stable project team (paragraph 2.7) m@ Growth in capital costs as plans for instruments were worked 
ith h 2.13 

m Critical success factors able to build on performance measures HOON NESSES ze ltsie) cla 
that are used for the original target station (paragraph 2.10 mw Estimation of operating costs (paragraphs 2.15 and 2.16 
and Figure 6) and Figure 8) 

m In response to the Gateway | review, the project team lm =Competition for direct procurement of components was mixed 
included a well-controlled contingency within the project's (Figure 10). For example, it was difficult to get companies ; 
budget (paragraph 2.11) interested in supplying steel for the relatively small amounts : 


(approximate value £10 million) required for the monolith — a 
structure required to ensure the safe working operation of the 
neutron target. 


Strong competition for the main building contract 


(paragraph 2.26) 
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: Energy Recovery Linac Prototype (ERLP) 


: Lead Research Council 


Council for the Central Laboratory of the Research Councils (CCLRC) 


- Location 


Daresbury Laboratory, Cheshire 


Facility description 


_ AGLS stands for Fourth Generation Light Source. It is a proposed 
facility for a low energy light source to complement table-top 


lasers, the Diamond synchrotron light source and the proposed 
European X-ray free electron lasers. The full project proposal will 


__ combine superconducting energy recovery linac and free electron 
_ laser technologies to deliver a suite of synchronised state of the art 


sources of synchrotron and free electron laser radiation covering the 


_ terahertz (THz) to soft X-ray regimes for dynamic studies of matter. 


The project being assembled at Daresbury is an Energy Recovery 


Linac Prototype (ERLP). It is designed to address some of the 


_ principal technical challenges that would be faced in a full AGLS 


facility. At the outset it was not intended that ERLP would support a 
long-term programme of scientific research. 


Phasing 


The full AGLS proposal had passed Gateway 1 (review of 
business case) when the decision to fund a prototype was taken. 
The ERLP is being built as a single phase project, which may yield 
a proposal to progress the full AGLS scheme further. 


Status at autumn 2006 


The ERLP is at an advanced build stage in the former nuclear 
structure facility experimental hall at Daresbury. The ERLP 


 photoinjector delivered first beam on 16 August 2006. The 


cryomodules are progressing through commissioning tests. Most of 
the sections of the beam transport system have been assembled. 


Shielding is in place. 


_ Capital cost and funding (ERLP) 


_ Capital cost 


£ million 

: Budget at project approval in 2003 12.9 
Latest estimate at autumn 2006 21.3 

: Variation +8.4 


(65.1 per cent) 


Reasons for variation — extension of duration of project from three 
to five years, full costing of prototype, addition of contingency 
provision, addition of extra equipment including a terawatt laser, 


funded by North West Development Agency. 
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Delivery timetable (ERLP) - demonstration of energy recovery 


At project approval in 2003 March 2006 
Latest estimate at autumn 2006 April 2007 
Variation +13 months 


Reasons for variation — project extended to provide more time for 
designing and testing of what is a prototype for a larger potential 
project. Also delays in delivery of key components (especially 
photo-injector ceramic, booster cavities and LINAC). 


Operating costs (ERLP) 


At project approval in 2003 None - all capitalised as 


Research and Development 


Latest estimate (October 2004) £0.5 million — £0.75 million per 


annum for exploitation if funded 


Variation Depends on level (if any) of 


exploitation programme 
Specification (ERLP) 
w high brightness electrons from photo-gun 
m beam diagnostics, test beamline and diagnostic area 


Hi superconducting linac delivering at least 30 MeV 
energy increase 


bunch length control in a compressor system 
2K cryogenics system 


electron beam dump 


spontaneous sources of radiation 


a 

& 

8 

M energy recovery transport line 

a 

m free electron laser with opportunities to explore seeding 
3a 


opportunities for studying source synchronisation 


These core components have been retained throughout the 
prototype’s development though the size of machine has been 
reduced, existing concrete and iron blocks have been reused as 
shielding rather than lead shielding, and maximum accessible 
electron energy has been reduced from 50 to 45 MeV. CCLRC 
have brought back into service disused experimental areas and 
unused infrastructure such as two radial cranes. 


Procurement strategy (ERLP) 
m Design by CCLRC Daresbury staff 
mw Direct procurement of individual components 


m Assembly/installation by CCLRC Daresbury staff 


Sources of funding £ million 

_ Large Facilities Capital Fund 10.1 
i CCLRC 8.0 
\ North West Development Agency Lay 
EU Framework 6 Programme 0.3 
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Energy Recovery Linac Prototype (ERLP) continued 


_ Critical Success Factors 


mw Build ERLP to a specification that allows key technical 
challenges of AGLS to be addressed 


m Demonstrate successful operation of the principal design 
concepts of 4GLS by addressing the key technical challenges 


Establish expertise in every 4GLS design topic 


Have active international collaborations in place for those 
challenges that are best addressed in this way 


: m Report on the results of addressing the challenges and the 
impact of these studies on the design of AGLS 


: Summary of Project-Specific Strengths 


_m Extensive testing of science case with international and 
| national scientific communities (paragraphs 1.13) 


| m lLevel and form of engagement with industry (paragraphs 1.16 
Bnd in7) 
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gm Flexibility and adaptability in response to cost pressures 
(paragraph 2.13) 


m Maximising re-use of existing equipment and contributions in 
kind to add value (paragraph 2.13) 

Summary of Project-Specific Issues 

m Limited time for initial costing of prototype (paragraph 2.13) 


m No Gateway reviews of the prototype — reliance on reviews 
of AGLS (paragraph 2.4) 


Subsequent capital cost increases (paragraph 2.13) 
Slippage in overall delivery date (paragraphs 2.20-2.21) 


Absence of contractual incentivisation of prompt delivery of 
components (paragraph 2.28) 
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Halley VI Antarctic Research Station 
Lead Research Council 
~ Natural Environment Research Council (NERC) 
Location 


: Brunt Ice Shelf, Antarctic. Managed from British Antarctic Survey 
_ Headquarters, Cambridge 


: Facility description 


_ The Halley Antarctic research station provides a unique location 


_ for monitoring climate, ozone and space weather. The main 
__ users of the station come from within the British Antarctic Survey 


__ (an institute of NERC). The project involves the construction of 
~ anew relocatable station - Halley VI - and the removal of the 


existing station, Halley V. Occupation of Halley V would become 


_ increasingly unsafe after 2010 due to the increased likelihood of 
_ the breaking and separation of the ice-shelf. 


: Status at autumn 2006 


: The project passed Gateway 3 (contract award) in August 
_ 2006 and contract was signed with the prime contractor in 
_ September 2006. 


_ Capital cost, decommissioning cost and funding 


8 Cost Halley VI Halley V Total 
construction decommissioning! 
£ million £ million £ million 
_ Budget at project 24.5 es) 34 
approval in 2003 
; Latest estimate at ES) 5 8.5 38 
- autumn 2006 
- Variation +4 


(+11.8 per cent) 


_ Note 1 —- Decommissioning costs are not treated by the Natural 


Environment Research Council as capital expenditure. 


Reason for variation - Tender price from prime contractor for 


~ building Halley VI was £16 million higher than expected. Project 
_ team reduced impact of higher tender price by reducing the size 


of Halley VI and combining the construction of Halley VI with the 
decommissioning of Halley V which will cut costs but will also 


_ curtail the science programme for a minimum of two years. 


_ Sources of funding £ million 
Large Facilities Capital Fund 20.0 
NERC (including British Antarctic Survey) 147 
Shortfall Sad 


_ Note - Shortfall may be made up from industrial sponsorship or 


38 


from British Antarctic Survey's own operating budget. NERC is 


providing a mix of capital and non-capital funding. The latter to 
cover decommissioning costs. 


eS CRS 
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Delivery timetable 


Halley VI 


start operations 


Halley V 
decommissioned 


Antarctic summer 


2009-10 


Antarctic summer 


2008-09 


At project approval 
in 2003 


Antarctic summer 
2009-10 


Antarctic summer 


2009-10 


Latest estimate at 
autumn 2006 


Variation + 12 months None 


Reason for variation — the time required to undertake the 
procurement and let contracts was longer than had been allowed 
for at project approval in 2003. 


Operating costs 


At project approval in 2003 Not provided in original 


business case 


Latest estimate at autumn 2006 ~—-.20 per cent reduction 


in staffing from Halley V levels 


Not identifiable — relative to 
business case 


Variation 


Specification 


Original 2003 Business Case specified that Halley VI was 
to accommodate a minimum of 16 personnel in the 
winter and a maximum of 52 personnel in the 
summer. Structures to be either relocatable and be 
able to be moved to a new site up to 20 kilometres 
away every 10 years, or to be located ata site 


25 kilometres from the edge of the ice shelf. 


Current Minimum accommodation specification has been 
maintained and structures will be relocatable. 
Initial design in 2004 envisaged that Halley VI 
would comprise 11 modules, as part of reducing 


costs this has now been reduced to eight modules. 


Procurement strategy 


B Initially envisaged that three shortlisted design companies 
would work with one of the same number of shortlisted 
construction companies to deliver a tender for design 


and build. 


m Only one compliant contractor expressed interest in the 
construction element. The procurement strategy was thus 
revised with each of the three designers working with the 
same construction company to firm up their proposals. 


m Once a preferred design was selected the construction 
company approached the market to cost the project, and the 
resulting estimate was not affordable. 


m Project was re-scoped and a contract was let for the 
construction of Halley VI and the demolition of Halley V. 
Design risk for Halley VI transferred to the contractor. Within 
the “target cost” construction contract, the main contractor 
will be paid actual costs plus a percentage fee. The contract 
includes pain/gain provisions to encourage the contractor to 
minimise actual costs. 
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_ Halley VI Antarctic Research Station continued 


Critical Success Factors Summary of Project-Specific Strengths 
_ Twelve critical success factors were identified in April 2004. m Recruitment of external project manger in 2003 has boosted 
_ They covered project timescale, minimum specification (e.g. project management skills and aided stability in project team | 
_ accommodation), fitness of facilities for research and science and (paragraphs 2.6 and 2.7) 


environmental impact. Examples of the success factors include: : 
m@ Project team have contained the level of cost growth by 


m= Halley VI facilities fit for Physical Science Division's (of the identifying and costing a wider range of options for delivering 
British Antarctic Survey) long-term monitoring and core the science objectives and by combining construction with 
science research. demolition of the existing Halley V station (paragraph 2.13) 

| @ Halley VI facilities are suitable to accommodate a Physical & Pain/gain share provisions in main contract (paragraph 2.28) 


Science Division’s “well found” laboratory. 


SERRE REST I EA IOS 
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| @ Halley VI facilities fit for gateway access to the interior for all oes cere (eos pecibelisstcs 


the science divisions of the British Antarctic Survey. m& Prior to Gateway 3, reviews had lacked independence as they 


"The construction and through life operation of Halley VI TENE DESIRE MEING SIS WEN eG Sette IU : 


complies with the requirements of the Antarctic m Expressions of interest were received from only one compliant 
Environmental Protocol. contractor (paragraph 2.25) 

_ The through life operating costs are significantly reduced, & Original tender to build Halley VI much higher than budget 
bench marked against Halley V. (paragraph 2.13) 

_ B® There should be minimal disruption to the science activity m Operating costs not specified in original business case 
on the Brunt Ice Shelf during the project i.e. the long-term (paragraph 2.15) 


monitoring datasets. 
: m Some science will be suspended whilst Halley VI is built 


_ In addition, a further thirteen “significant success factors that are (paragraph 2.13) 
_ not specifically measurable but that would enhance the value of 


this project” were identified in April 2004. These included: Mm Slippage in delivery date of Halley VI (paragraphs 2.20 and 


: 2.21 and Figure 9). Further delay would increase the risk that 
~ m@ A reduction in the requirement for manual and mechanical the project would not be completed before any breaking and 
snow management. separation of the ice-shelf where the current Halley station is 


m A reduction in the multiple handling and manual handling esl osbipsiees Keio Coe) 


of stores. 
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_ HECToR High End Computing Servic 


sees 


STERN i 


' Lead Research Council 


: Engineering and Physical Sciences Research Council (EPSRC) 


Location 


Dependant on tendered proposals 


_ Facility description 


_ HECTOR is the next generation of high performance computer 
service. It will replace the CSAR service at Manchester University 


which ended in June 2006, and the HPCx service operating at 
Daresbury in Cheshire, which is due to close at the end of 2008. 
It will provide a high capability computing resource for the UK 


scientific community, with the ability to perform complex and inter- 


related calculations on a single machine. This meets a different 


"requirement to the high capacity computing offered by Grid-based 


solutions. 


Phasing 


HECToR will be implemented in three phases, with each phase 
lasting two years and representing a doubling of the initial 
sustained capability. 


Status at autumn 2006 


_ At November 2006, the preferred hardware provider, the 


computational science and engineering support provider, and 


__ tenders for the facilities management provider had been selected. 


_ Capital cost and funding 


~ Capital cost £ million 
Budget at project approval in 2004 65.0 

_ Latest estimate at autumn 2006 59.4 

_ Variation 5.6 


(8.6 per cent lower) 


Reasons for variation — Cost reduction mainly due to project 
procuring hardware at a lower cost than expected. 


Sources of funding £ million 
_ Large Facilities Capital Fund 5210 
EPSRC 9.0 
NERC 33 
BBSRC 0.7 


k 
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__ At project approval in 2004 
; Latest estimate at autumn 2006 
~ Variation 


Reasons for variation 


Delivery timetable 

December 2006 
September 2007 
+ 9 months 


Project delayed to 
accommodate availability 
of Large Facilities Capital 
Fund support. 
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Operating costs 


£5.4 million per annum 
over six years 


At project approval in 2004 


Latest estimate at autumn 2006 £8.2 million per annum over 
six years subject to finalisation 
of contract for service provision. 
Estimate is based on predicted 
rather than final prices as 

at autumn 2006 contracts for 
service provision had not 


been finalised. 


+£2.8 million per annum 
(+51.9 per cent) 


Variation 


Reasons for variation Increased electricity prices. 
Increased estimated cost per 
staff member on computational 


support and engineering. 


Specification 


When it was approved in 2004, the project was specified in 
terms of an initial peak capability of 50-100 teraflops, doubling 
at each upgrade. The specification was changed in 2005 to a 
requirement that initial sustained capability should be at least 2.5 
times that of the HPCx service, rising by a multiple of 2.5 at each 
upgrade. This has resulted in a nominal initial peak performance 
within the range initially specified for Phase 1 and significantly 
above that initially specified for Phase Il. The change from peak 
to sustained capability was driven by an understanding that the 
latter was a more meaningful requirement for the applications of 
the user community. 


Procurement strategy 


™ Three contracts — supply of hardware, computational support 
and engineering (CSE) and facilities management including 
accommodation 


m Responsibilities for the hardware contract will pass to the 
facilities provider — CSE contract stays with EPSRC 

Critical Success Factors 

Developed following Gateway 1 and in preparation for Gateway 2 


m world-class scientific output as measured by bibliometrics and 
international reviews 


M greater scientific productivity as measured by code 
performance, slowdown, utilisation and availability 


training support for graduates and postdocs as measured by 
numbers of such users and annual survey 


@ increase in UK computational science and engineering skills 
base as measured by number of staff in CSE contract 


increased collaboration with industry as measured by number 
of research grants with industrial collaboration 


m strengthen UK’s position for world-class science as measured 
by position in TOP500 list of supercomputers 
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_ HECToR High End Computing Service continued 


Strong governance structures developed during the project's Eo 
evolution (paragraph 2.5) 


_ & Critical success factors relatively well developed (paragraph S 
2.10 and Figure 6) 


m Extensive market engagement yielding strong interest from 


hardware suppliers (paragraph 2.26) = 


f Summary of Project-Specific Strengths Summary of Project-Specific Issues 


APPENDIX THREE 
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Increases in estimates of operating costs (Figure 8 and 
paragraph 2.17) 


Handling of depreciation charges (paragraphs 2.3 and 2.14) 


Limited engagement with industry to realise exploitation 
potential (paragraphs 1.16 and 1.17) 
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Few tenderers for computational support, engineering and 
facilities management contracts (paragraph 2.26 and Figure 10) 
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Muon lonisation Cooling Experiment (MICE) - Phase 1 


: Lead Research Council 


; Particle Physics and Astronomy Research Council (PPARC) holds 
_ the funding from the Large Facilities Capital Fund. The experiment 


is hosted by the Council for the Central Laboratory of the Research 
Councils (CCLRC) 


Location 


Harwell Science and Innovation Campus, Oxfordshire 


Facility description 


The Muon lonisation Cooling Experiment (MICE) is a step towards 
the possible creation of a factory capable of producing a very 
intense and focused beam of neutrinos. Such a factory would 

aid the understanding of the properties of neutrinos which are 
one of the fundamental particles'* which make up the universe. 
MICE seeks to demonstrate that “muon cooling” — making a 
tightly focused muon beam - is possible through a process of 
ionisation. It is thought that such a breakthrough could improve 
the performance of a future neutrino factory between four and ten 
times. The project is being taken forward through an international 
collaboration including input from continental Europe, North 
America and Japan. 


Phasing 


m Phase 1 — Build a characterised muon beam on the 
ISIS neutron scattering facility at Rutherford Appleton 
Laboratory, Harwell. 


m Phase 2 — Implement the full MICE experiment. 


Status at autumn 2006 


Phase 1 has been approved. The project passed through a 
combined Gateway 2 and 3 (procurement strategy and contract 
award) in December 2004. By autumn 2006 funding for Phase 2 
had not been confirmed. 


Capital cost and funding (Phase 1 only) 


Capital cost UK only Total for 

UK and 
international 

project 
: £ million £ million 
| Budget at project approval in 2004 OF 227 
‘ Latest estimate at autumn 2006 OF 22 
Variation None None 
Sources of funding £ million 
Large Facilities Capital Fund es 
CCLRC 1x 
Particle Physics and Astronomy Research Council 0.9 
International partners 1320 
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Delivery timetable (Phase 1) 


At project approval in 2004 By end of 2006-07 


Latest estimate at autumn 2006 November 2007 


Variation +8 months 


Reasons for variation — Much of the work on MICE can only 

be undertaken when the host ISIS facility is shutdown. A major 
planned shutdown of ISIS has been put back which will delay 
completion of MICE. Within the MICE project progress has been 
slower than planned due to problems in getting staff needed for 
the project. However, the delay resulting from staffing problems 
has been less than that caused by the delay to the ISIS shutdown. 


Operating costs (Phase 1 and 2 — UK only) 


Total lifetime UK costs of exploiting MICE are expected to be in 
the region of £5 million. 


At project approval in 2004 £1.6 million per annum for 


three years 


Latest estimate at autumn 2006 £1.8 million per annum for 


three years 


Variation +£0.2 million per annum for 


three years (+12 per cent) 


Specification (Phase 1) 


Original — At project approval in 2004, the UK’s planned 
contributions to the building of a characterised muon 
beam facility were to include: installation and 
commissioning of the MICE Muon Beam on ISIS; 
execution of a programme of essential research and 
development; construction, installation and 
commissioning of the tracker system required to meet 
the first measurement phase of MICE 


Current No major change from above 


Procurement strategy 
mw Experiment will be located in an existing hall at the ISIS facility 


m Direct procurement of components, such as magnets 
and solenoid cryogenic. For the latter there was limited 
competition 


m Use of some refurbished components and materials 


m Contributions in kind from international collaborators in terms 
of equipment and staff time 


Installation and assembly by staff from CCLRC and 
UK universities 


PEE PRG Bak BEER 


_ Critical Success Factors 


| The 2003 Business Case identified “three broad key criteria for 


- For the international collaboration taking forward the MICE 
experiment, to demonstrate that an ionisation cooling channel 
can be designed and built, and its performance understood. 
The key deliverable is publication of the results of the cooling 
demonstration. 


m For the UK MICE (i.e. CCLRC, PPARC and universities) 
collaboration, to demonstrate that it can lead and host 
advanced accelerator research and development. The key 
deliverables are the provision of the muon beam as specified 
and the UK contribution to MICE, and the participation and 
the leadership of the analysis of the results. 


: m For the CCLRC, to demonstrate that it can host and manage 
an international collaborative project in advanced accelerator 
research and development. The key deliverables are the 
performance and reliability of muon beam and technical 
support to MICE, and the successful delivery of the project 
within the agreed cost, time and quality envelope. 
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: success” for the MICE project reflecting the different groups involved: 


Muon lonisation Cooling Experiment (MICE) - Phase 1 continued 
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Summary of Project-Specific Strengths 


Extensive engagement and involvement of international and 
national scientific communities (paragraphs 1.12 and 1.20) 


Project team active in overseeing and supporting contribution 
of overseas partners (paragraph 2.24) 


Re-use of existing equipment and materials from the UK and 
overseas (paragraph 2.13) 


Stability in project team (paragraph 2.7) 


In response to the Gateway 1 review, the project team 
identified in detail and costed the various tasks necessary to 
complete the project (paragraph 2.11) 


Summary of Project-Specific Issues 


w Delay in shutdown of the host ISIS facility has pushed back the 


completion date of Phase 1 (Figure 9) 


Phase 2 was not funded by autumn 2006 and thus there 
remained a risk that project may not be completed 
(Figure 7) 


If Phases 1 and 2 are not completed until 2010 it could 
significantly compromise the prime objective of MICE of 
informing the design of a neutrino factory (paragraph 2.23) 
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_ Lead Research Council 


Medical Research Council (MRC) 


Location 


’ Cambridge — Addenbrooke’s Hospital Campus 


_ Facility description 


_ The Laboratory of Molecular Biology is an institute of the Medical 
- Research Council which opened in 1962 on the Addenbrooke’s 
Hospital site in Cambridge. It is widely acknowledged as one 

_ of the world’s leading biochemical laboratories with strengths 

_ in structural biology, cell biology and neurobiology, and strong 


groups in immunology, cancer biology and biotechnology. The 


_ building was extended in 1968 and a new block was added in 
_ 1970. The project is to provide new premises for the institute to 
tackle overcrowding and replace buildings nearing the end of 

_ their planned lifetime. 


Phasing 


The project is planned as a single phase construction project. 


Status at autumn 2006 


Capital cost 


An outline brief had been converted into a concept design and 
the project team submitted its procurement strategy for Gateway 2 
review in November 2006. 


Capital cost and funding 


£ million 

Budget at project approval in 2005 155 

_ Latest estimate at autumn 2006 164 
Variation +9 


(+5.8 per cent) 


Reasons for variation — inflationary pressures as project delivery 


has slipped since approval 


Sources of funding £ million 
Large Facilities Capital Fund 67.0 
_ Medical Research Council 88.0 


Delivery timetable 


; At project approval in 2005 June 2009 
' Latest estimate at autumn 2006 May 2011 
- Variation +23 months 
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Reasons for variation — the developer of the wider site (where 
the Laboratory is to be located) has not secured planning 
permission as quickly as expected. Also extended operational 


_ commissioning period 
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Operating costs 


At project approval in 2005 


Latest estimate at autumn 2006 


Variation 


Specification 


Original proposal was for 24,000m? building. By the time of 
business case approval in 2005 the design was for 27,000m? 
building. By autumn 2006 the design had developed and the 
building had increased to 30,438m? to allow for more plant space. 
Dedicated areas for industrial collaboration and School of Clinical 
Studies occupation have been omitted but with some flexibility to 
incorporate the former into the “Director’s wing” and the latter to be 
incorporated if university funding is forthcoming. 


Procurement strategy 


Critical Success Factors 


No critical success factors were set out in the business case 
submitted to MRC council. 


Summary of Project-Specific Strengths 


Extra running costs of 

£1.2 million per year relative to 
current building. It is also 
expected that the new building 
would be cheaper to maintain 
(by an unspecified amount) 

for the first ten years 


No later estimate available 


Not applicable 


In-house preparation of outline brief 


External consultants employed to develop concept design 


Different external team used to develop detailed design 
Tendering of contract to build to detailed design 


Consideration being given to the construction contractor 
taking over responsibility for the design team 


Wide agreement on the scientific strength of the institute and 
the need for a new building (paragraph 1.12) 


Detailed consideration of some lifetime costs e.g. interstitial 
floors and chilling options (paragraph 2.20) 
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Value engineering to contain costs (paragraph 2.13) 


Summary of Project-Specific Issues 


Initial cost estimate at prioritisation was £55 million below the 
budget approved in 2005 (paragraph 1.18) 


Since project approval, forecast capital cost has increased 
by a further £9 million. This has been driven by inflationary 
pressures caused by a delay in the project. The developer 
of the wider site (where the Laboratory will be located) has 
taken longer than expected to secure planning permission 
(paragraph 2.13 and Figure 9). 


Loss of specific space for industrial collaborations 
(paragraph 1.17) 


Gateway 1 missed (paragraph 2.4) 


No critical success factors (paragraph 2.9) 
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Institute for Animal Health 


_ Lead Research Council 


Biotechnology and Biological Sciences Research Council (BBSRC) 


Location 


Pirbright Laboratory — Pirbright, Surrey 


Facility description 


Pirbright, is one of three locations where the Institute for Animal 
Health has laboratories. The Pirbright laboratory has a remit to 
carry out research, diagnostics and surveillance on epizootic 
(fast spreading) viral diseases of farm animals, including foot and 
mouth disease. The new facility will provide modern, combined 
laboratory buildings for staff from the Institute and the Virology 
Department of the Veterinary Laboratories Agency currently 
located at Weybridge, Surrey. The Virology Department is 
responsible for research into endemic and epizootic viral diseases 
in farm animals, poultry and horses, and some work into diseases 
of importance to public health. 


Phasing 


m Advance works — preparation of the site and some initial 
works for the main building 


@ Main building programme — includes a new main laboratory 
research complex; an avian isolation unit; refurbishment of some 
existing buildings and some supporting infrastructure works 


Status at autumn 2006 (main building programme) 


The project went through Gateway 2 (Procurement Strategy) in 
February 2006 and was rated as ‘Green’. The principal contractor 
for the main building programme was appointed in September 
2006 and the contract was signed in November 2006. 


Capital cost and funding (for both Phases) 


Capital cost £ million 
Budget at project approval in February 2006 121 
Latest estimate at autumn 2006 WA 

_ Variation None 
Sources of funding £ million 
Large Facilities Capital Fund 31 
BBSRC 23 

_ Department for Environment, Food and Rural Affairs 67 
Delivery timetable (completion of main laboratory building - part 
of main building programme) 
At project approval in February 2006 November 2009 

_ Latest estimate at autumn 2006 December 2009 

_ Variation +1 month 

_ Reasons for variation — additional time needed to implement 
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Operating costs 


At project approval 
in February 2006 


Latest estimate at 


autumn 2006 


Variation 


Operating costs not specified in 
original business case prepared in 
January 2004. Whole life costs for 
the programme (including capital and 
operating costs, based on a 

60-year life) were calculated at 
around £5,900 million in 

January 2006 


Being reviewed, due to be 


delivered end November 2006 
Not applicable 


Specification 


Original 


Current 


The original business case in January 2004 proposed 
a new laboratory to accommodate up to 280 staff with 
a gross internal area of 28,500m?, including a 
science area of 14,800m’. By project approval in 
February 2006 the Laboratory was designed for 

223 staff with the gross internal area reduced to 
13,900m?2 and the science area to 6,200m?. 


No change from February 2006. 


Procurement strategy 


The construction contract for the building within the advanced 
works phase is controlled through a traditional contract in 
which the Institute retains much of the design risk. 


A two-stage design and construction procurement route was 
selected for the main building programme. 


During Stage 1 the selected contractor will work with the 
client’s design team to review and develop the design and 
arrive at a contract price. If the contract price is accepted, 
then the design responsibility is then transferred to 

the contractor. 


The client’s design team will maintain their contract with the 
Institute for Animal Health rather than being subcontracted to 
the successful contractor. 


Critical Success Factors 
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Meet government and statutory requirements. Transfer of 
current operations at the Institute for Animal Health and 
the Veterinary Laboratories Agency to the new facility with 
minimal disruption 


Retention of reference laboratory status 


Provision of service and contingency capacity during 
redevelopment project 


Enhanced physical security and biosecurity. No actual or 
potential breaches of disease security or health and safety 
requirements as a result of the new procedures 


Improved quality assurance of operations 


Deliver the programme on time, within budget and to a 
sustainable business plan. 
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Institute for Animal Health continued 


| 


Summary of Project-Specific Strengths 


w New enhanced project team and management structure put in 
place to prepare and deliver procurement strategy 
(paragraph 2.7) 


m Benchmark data used to inform recosting of project post 
business case (paragraph 2.1 1) 
Summary of Project-Specific Issues 


w Initial project team had insufficient experience of managing 
large complex projects (paragraph 2.7) 


m Fast track approach was adopted to preparing the project's 
business case. Project was not ready when it went through 
Gateway 1 in January 2004 and the period to move from 
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business case to procurement strategy for the Pirbright 
redevelopment was extended. It took two years to prepare 
the procurement strategy, leading to a delay in the planned 
completion date of the new laboratory by a year 
(paragraph 1.18) 


Since being prioritised for support from the Large Facilities 
Capital Fund in 2004 the project has been rescoped. The : 
new laboratory will now house 25 per cent less staff than 
originally envisaged when £121 million was first earmarked 
to the project in 2004. The project's principal sponsors 
have confirmed the new laboratory will meet their needs 


(paragraph 1.18) 


Operating costs were not included in original business case at 
Gateway 1 (paragraph 2.15) 
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Research Complex and Essential Infrastructure 


Lead Research Council 


Medical Research Council (MRC) on behalf of other councils. 
| MRC hold the funding from the Large Facilities Capital Fund and 
will operate and manage the Complex. 


Location 


| Harwell Science and Innovation Campus, Oxfordshire 


Facility description and phasing 


The Research Complex will comprise laboratory and office space 
for visiting users of Diamond, ISIS and other facilities at Harwell. 
The Complex is intended to be multi-disciplinary in order to 

_ encourage collaborative working between physical scientists and 
life scientists. 


_ The essential infrastructure is a programme of works designed to 
improve the facilities at Harwell to meet the increased demand 
that Diamond and ISIS 2 will place on existing services. 


_ Status at autumn 2006 (the Research Complex) 


_ The project passed through Gateway 1 in March 2006. Funding 
from the Large Facilities Capital Fund was formally announced in 
_ August 2006. 


Capital cost and funding 


Capital cost Research Essential 
Complex Infrastructure 
£ million £ million 
Budget at project approval 26.4 Tol 
in March 2006 
Latest estimate at autumn 2006 26.4 8.3 
_ Variation @) + 1.2 


(+16.9 per cent) 


Reasons for variation — cost of upgrading power has increased by 
£0.4 million and the costs of improvements to the restaurant and 

_ to the kitchen are likely to be in the region of £0.8 million higher 
than budget 


_ Sources of funding Research Essential 
Complex Infrastructure 
£ million £ million 
Large Facilities Capital Fund 26.4 6.0 
~ CCLRC (at March 2006) ea 
_ Delivery timetable (Research Complex) 
At project approval in March 2006 March 2009 
_ Latest estimate at autumn 2006 June 2009 
Variation +3 months 


Reasons for variation — The Programme Board decided that 
additional time was required to engage users whilst the facility 
was being designed. As a result, the date for Gateway 2 was 
moved from June 2006 to November 2006 and the completion 
date was also put back. 
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Operating costs (Research Complex) 


At project approval in March 2006 
Latest estimate at autumn 2006 


Variation 


Specification (Research Complex) 


Original 


Current 


Procurement strategy (Research Complex) 


Subject to Gateway 2 review in January 2007. 


Critical Success Factors (Research Complex) 


Summary of Project-Specific Strengths 


£1.64 million per annum 
Not yet reviewed 


Not applicable 


The business case approved in 2006 specified that 
the area required for the research complex is 
4,756m? net internal area and 5,658m* gross. 
Facilities must be able to cope with diverse 
research requirements. 


No change from above 


a 
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Creation of an exciting place to work. 


The provision of facilities that will create world-class research 
and world-class personnel/scientists. 


Integration and interaction (synergy) between 
scientific disciplines. 


Rutherford Appleton Laboratory (i.e. Harwell) operations 
continue unaffected by the works. 


Diamond achieves its scientific potential in delivering science 
by the in-house and user communities. 


The project is completed on time, within budget, to the 
required quality and done safely. 


Specification of critical success factors and performance 
measures in the project’s business case (paragraph 2.10 
and Figure 6) 


In preparing the project’s science and business case, the 
project team identified and costed a range of options 
for varying the size and equipment within the building. 
Benchmarking data was used to inform costings 
(paragraph 2.11) 


Business case identified options for funding operating costs 
which will fall to a number of Research Councils 
(paragraph 2.14) 


Stability in project team (paragraph 2.7) : 


Summary of Project-Specific Issues 


The funding allocation from the Large Facilities Capital Fund 
has driven the timing of the project with the Complex planned 
to be completed in 2009 some two years after the Diamond 
Synchrotron is likely to open (Figure 9) 
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ENDNOTES 


1 The Green Book: Appraisal and Evaluation in Central 
Government, HM Treasury Guidance. 


2 The Office of Government Commerce expects a 
PRINCE2 project to have the following characteristics: 
a finite and defined life cycle; defined and measurable 
business products (or objectives); a corresponding set 
of activities to achieve the business products; a defined 
amount of resources; an organisation structure, with 
defined responsibilities, to manage the project. http:// 
www.ogc.gov.uk/methods_prince_2_overview.asp 


3. The new Council will not be responsible for all large 
facilities. For example, the Biotechnology and Biological 
Sciences Research Council and its partners will remain 
responsible for the redevelopment and running of the 
laboratory located at the Institute for Animal Health. 


4 Setting Priorities for Large Research Facility Projects 
supported by the National Science Foundation, Committee 
on Setting Priorities for National Science Foundation- 
Sponsored, Large Research Facility Projects, National 
Academies Press, Washington, D.C. 2004. 

A Framework for the Evaluation, Funding and Oversight of 
Canadian Major Science Investment: Draft Discussion Paper 
Office of the National Science Advisor, Ottawa, 2005. 
Final Report of the National Research Infrastructure Task 
Force, Australian Government Department of Education, 
Science and Training, Canberra 2004. 


5 European Road Map for Research Infrastructures, 
Report 2006, European Strategy Forum on Research 
Infrastructure (www.cordis.europa.eu/esfri/). 


6 Delivery Plan 2005-2008 CCLRC, May 2005. 


7 Future Access to Neutron Sources: A Strategy for the 
UK, CCLRC, March 2006. 


8 — Science and Innovation Investment Framework 2004- 
2014: Next Steps, HM Treasury, Department for Education 
and Skills, Department of Trade and Industry, Department 

of Health, March 2006. 


9 As part of an on-going programme of work to better 
understand what drives the performance of major defence 
projects, the NAO have developed a gold standard of 
project control. A summary of the standard is at 
www.naodefencevfm.org/downloads/pdf/gold_standard_ 
poster.pdf. 


10 The new performance management system was set 
out in Science Budget Allocations 2005-06 to 2007-08 
Department of Trade and Industry, May 2005 
(www.dti.gov.uk/files/file1 4994. pdf). 


11. None of the projects prioritised from the 2005 road 
map had been approved by autumn 2006. 


12 In particle physics, an elementary particle or 
fundamental particle are those subatomic particles not 
known to be compositional — i.e. made up of smaller 
particles. Current particle theories claim that such 
indivisible particles are the “fundamental particles” from 
which all larger particles in the universe are made. Both 
neutrinos and muons are fundamental particles. 
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